JP7663374B2 - Display device - Google Patents

Description

This disclosure relates to a display device.

Patent document 1 discloses a technology that detects highlight scenes by detecting changes in the audio signal level, such as loud cheers from the audience or loud voices from the announcer, from a video signal.

JP 2003-134415 A

The technology disclosed in Patent Document 1 identifies highlight scenes based on changes in the audio signal level included in the video signal, so there is a risk that scenes that do not actually excite the viewer may be detected as highlight scenes. One aspect of the present disclosure aims to provide a display device that can identify scenes that actually excite the viewer as highlight scenes.

A display device according to one aspect of the present disclosure includes a display unit that displays a moving image based on a received video signal, an acquisition unit that acquires changes in a viewer watching the moving image displayed on the display unit, and an identification unit that identifies a highlight scene of the moving image based on the changes in the viewer acquired by the acquisition unit.

FIG. 1 is a diagram illustrating an example of a configuration of a display device according to a first embodiment. 5 is a flowchart showing an example of an operation of the display device according to the first embodiment. 11 is a flowchart illustrating an example of a process for acquiring a physical quantity resulting from a viewer, which indicates a change in the viewer. FIG. 11 is a diagram illustrating an example of a configuration of a display device according to a second embodiment. 10 is a flowchart showing an example of an operation of the display device according to the second embodiment. FIG. 13 is a diagram illustrating an example of a configuration of a display device according to a third embodiment. 13 is a flowchart showing an example of an operation of the display device according to the third embodiment. FIG. 13 is a diagram illustrating an example of a configuration of a display device according to a fourth embodiment. 13 is a flowchart showing an example of the operation of the display device according to the fourth embodiment. FIG. 13 is a diagram illustrating an example of a configuration of a display device according to a fifth embodiment. 13 is a flowchart showing an example of the operation of the display device according to the fifth embodiment.

First Embodiment
A first embodiment will be described with reference to Figures 1 to 3. In the drawings, the same or equivalent elements are given the same reference numerals, and duplicated explanations will be omitted.

FIG. 1 is a diagram showing an example of the configuration of a display device 100. The display device 100 includes a content input unit 101, a video processing unit 102, a display unit 103, a storage unit 104, an acquisition unit 105, an identification unit 106, a playback unit 107, and a video switching unit 108. The display device 100 is, for example, a television receiver. The display device 100 also includes an operator 109. The operator 109 is operated by a viewer of the display device 100 and accepts operations by the viewer. The operator 109 is, for example, an operation button or the like, and may be a remote control.

The content input unit 101 receives a video signal 111. The content input unit 101 is, for example, a broadcast wave receiving device, a packaged media playback device, or a VOD (Video On Demand) device. For example, the video signal 111 is a video signal of a program broadcast on digital television. The video signal 111 may also be a video signal of a program acquired from a network such as the Internet. The video signal 111 may also be a video signal stored in an external device such as a video camera or a recording device connected to the display device 100. Furthermore, the video signal 111 includes sounds such as voice, music, or sound effects.

The video processing unit 102 plays back the video image 112 based on the video signal 111 received by the content input unit 101.

The display unit 103 displays a moving image. Specifically, the display unit 103 displays a moving image 112 based on a video signal 111 received by the content input unit 101. Alternatively, the display unit 103 displays a moving image 114 based on a video signal 113 corresponding to a highlight scene. For example, the display unit 103 is a liquid crystal panel or an organic EL (Electro-luminescence) panel.

The storage unit 104 stores the video signal 111 and is, for example, a storage device such as a random access memory (RAM), a hard disk drive, or a flash memory. The storage unit 104 stores a video signal 113 that corresponds to a highlight scene, which is part of the video signal 111.

The acquisition unit 105 acquires changes in the viewer viewing the moving image displayed on the display unit 103. The acquisition unit 105 includes a sensor that detects physical quantities resulting from the viewer that indicate changes in the viewer. The acquisition unit 105 according to this embodiment includes a microphone 121, an image sensor 122, a vibration meter 123, and an infrared camera 124 as sensors that detect physical quantities resulting from the viewer.

The microphone 121 acquires sounds generated by the viewer. The microphone 121 is installed on the display device 100 so as to acquire sounds from the direction in which the display unit 103 faces. The image sensor 122 captures images of the viewer and generates an image. An RGB camera equipped with the image sensor 122 is installed on the display device 100 so as to capture images of the viewer facing the display unit 103. The vibration meter 123 measures vibrations generated by the viewer. As an example, the vibration meter 123 is installed on the display device 100. The infrared camera 124 acquires temperature information. The temperature information indicates temperatures at multiple positions within a predetermined area facing the infrared camera 124. For example, the infrared camera 124 includes a multi-lens infrared sensor arranged in an array, and measures temperatures at multiple positions. The infrared camera 124 is installed on the display device 100 so as to measure the temperature of the viewer facing the display unit 103. This allows the infrared camera 124 to obtain temperature information indicating the viewer's body temperature, corresponding to the viewer's position included in the captured image captured by the imaging element 122.

The acquisition unit 105 further includes a volume change detection unit 131, a movement detection unit 132, a vibration change detection unit 133, and a body temperature change detection unit 134. The volume change detection unit 131 detects the amount of change in the volume acquired by the microphone 121. The movement detection unit 132 detects the amount of change in the movement of the image of the viewer included in the captured image based on the image processing result of the captured image generated by the image sensor 122. The vibration change detection unit 133 detects the amount of change in the vibration measured by the vibration meter 123. The body temperature change detection unit 134 detects the amount of change in the temperature measured by the infrared camera 124.

The identification unit 106 identifies highlight scenes of the video 112 based on the video signal 111, based on the changes in the viewer acquired by the acquisition unit 105.

The playback unit 107 plays back a video image 114 based on a video signal 113 corresponding to a highlight scene from the video signal 111 in response to a viewer's operation of the control 109.

The video switching unit 108 switches between the video 112 based on the video signal 111 and the video 114 based on the video signal 113 corresponding to the highlight scene, and outputs the video to the display unit 103, thereby displaying the video 114 based on the video signal 113 corresponding to the highlight scene on the display unit 103. For example, when the playback unit 107 plays back the video 114 based on the video signal 113 corresponding to the highlight scene, the video switching unit 108 displays the video 114 based on the video signal 113 corresponding to the highlight scene on the display unit 103. On the other hand, for example, when the playback unit 107 does not play back the video 114 based on the video signal 113 corresponding to the highlight scene, the video switching unit 108 outputs the video 112 based on the video signal 111 to the display unit 103, thereby displaying the video 112 based on the video signal 111.

Next, the operation of the display device 100 will be described with reference to FIG. 2. FIG. 2 is a flowchart showing an example of the operation of the display device 100. When the content input unit 101 receives a video signal 111, the display device 100 starts the process of step S201 illustrated in FIG. 2.

In step S201, the video processing unit 102 plays a moving image 112 based on the video signal 111 received by the content input unit 101.

In step S202, the acquisition unit 105 acquires a physical quantity resulting from the viewer that indicates a change in the viewer. The process of step S202 will be described in detail later.

In step S203, the identification unit 106 determines whether the physical quantity generated from the viewer satisfies a predetermined highlight condition. For example, the highlight condition is that the physical quantity generated from the viewer acquired by the acquisition unit 105 is equal to or greater than a predetermined threshold.

If the change in the viewer does not satisfy the predetermined highlight condition in step S203, the determination unit 106 returns the process to step S202. In other words, the determination unit 106 repeats the processes of steps S202 and S203 until the physical quantity generated by the viewer satisfies the predetermined highlight condition.

On the other hand, if the physical quantity generated from the viewer satisfies the predetermined highlight condition in step S203, the identification unit 106 identifies, in step S204, the scene that was being played when the physical quantity generated from the viewer satisfied the highlight condition as a highlight scene. If the physical quantity generated from the viewer acquired by the acquisition unit 105 becomes equal to or greater than a predetermined threshold value, and the physical quantity does not become equal to or greater than the threshold value within a predetermined time, the identification unit 106 determines that the highlight scene has ended.

Furthermore, when the physical quantity generated by the viewer acquired by the acquisition unit 105 becomes equal to or greater than a predetermined first threshold and then becomes equal to or greater than a second threshold that exceeds the first threshold, the identification unit 106 may identify as a highlight scene a scene from when the physical quantity becomes equal to or greater than the second threshold. Then, when the physical quantity does not become equal to or greater than the second threshold within a predetermined time after the physical quantity becomes equal to or greater than the second threshold, the identification unit 106 may determine that the highlight scene has ended.

In step S205, the identification unit 106 stores the video signal 113 corresponding to the identified highlight scene from the video signal 111 in the storage unit 104.

In step S206, the playback unit 107 determines whether or not an instruction to play back a highlight scene by a viewer's operation has been received. If the playback unit 107 has not received an instruction to play back a highlight scene by a viewer's operation in step S206, the playback unit 107 returns to the process in step S202. In other words, the playback unit 107 repeats the processes in steps S202 to S206 until an instruction to play back a highlight scene by a viewer's operation is received. On the other hand, if the playback unit 107 has received an instruction to play back a highlight scene by a viewer's operation in step S206, the playback unit 107 plays back the video 114 based on the video signal 113 corresponding to the highlight scene stored in the storage unit 104 in step S207.

In step S208, the video switching unit 108 switches between the video 112 based on the video signal 111 and the video 114 played in step S207, and outputs the switched video to the display unit 103, thereby causing the display unit 103 to display the video 114 based on the video signal 113 corresponding to the highlight scene. By performing the processes in steps S206 to S208, the display device 100 can allow the viewer to view the highlight scene at the timing desired by the viewer.

In step S209, the playback unit 107 determines whether or not playback of the video 114 based on the video signal 113 corresponding to the highlight scene has ended. If playback of the video 114 based on the video signal 113 corresponding to the highlight scene has ended in step S209, the video switching unit 108 in step S210 outputs the video 112 based on the video signal 111 to the display unit 103, thereby causing the video 112 based on the received video signal 111 to be displayed on the display unit 103. This allows the display device 100 to allow the viewer to continue viewing the video 112 based on the received video signal 111.

Next, the process of step S202 for acquiring the physical quantities generated by the viewer will be described in detail with reference to FIG. 3. FIG. 3 is a flowchart showing an example of the process of step S202.

In step S301, the volume change detection unit 131 acquires the sound generated by the viewer and acquired by the microphone 121. Here, since the sound contained in the video signal 111 is output from the display device 100, the microphone 121 acquires the sound generated by the viewer as well as the sound generated by the audio signal contained in the video signal 111. Therefore, the volume change detection unit 131 cancels the component of the audio signal contained in the video signal 111 from the component of the audio signal input to the microphone 121 by superimposing a signal of the opposite phase to the audio signal contained in the video signal 111 on the audio signal input to the microphone 121. In this way, the volume change detection unit 131 can acquire the sound generated by the viewer and acquired by the microphone 121 from the audio signal input to the microphone 121.

In step S302, the volume change detection unit 131 detects the amount of change in the volume acquired by the microphone 121 over a predetermined period of time.

In step S303, the motion detection unit 132 detects the image of the viewer in the captured image generated by the image sensor 122. For example, the motion detection unit 132 detects a face area from the captured image, and detects the image of the viewer based on the detected face area. In step S304, the motion detection unit 132 detects the amount of change in the movement of the image of the viewer in the captured image. For example, the motion detection unit 132 detects the amount of change in the position, size, etc. of the image of the viewer included in the captured image as the amount of change in the movement of the image of the viewer.

In step S305, the vibration change detection unit 133 acquires the output value of the vibration meter 123. In step S306, the vibration change detection unit 133 detects the amount of change in the vibration of the display device 100 based on the output value of the vibration meter 123. For example, the vibration change detection unit 133 detects the amount of change in the frequency, the amount of change in the amplitude, etc. of the vibration measured by the vibration meter 123.

In step S307, the body temperature change detection unit 134 acquires the temperature information acquired by the infrared camera 124.

In step S308, the body temperature change detection unit 134 detects the amount of change in the viewer's body temperature based on the temperature information acquired by the infrared camera 124. For example, the body temperature change detection unit 134 detects the amount of change in temperature indicated by the temperature information corresponding to the position of the viewer included in the captured image captured by the image sensor 122 at a predetermined time.

The amount of change in sound volume detected in step S302, the amount of change in the movement of the image of the viewer in the captured image detected in step S304, the amount of change in the vibration of the display device 100 detected in step S306, and the amount of change in the viewer's body temperature detected in step S308 are each amounts of change in physical quantities arising from the viewer. The more excited and lively the viewer becomes while watching the moving image 112 based on the video signal 111, the greater the amount of change in physical quantities arising from the viewer becomes compared to other scenes. Therefore, when the viewer is watching a highlight scene, there is a possibility that the amount of change in physical quantities arising from the viewer will be greater than in other scenes.

For example, when a viewer is watching a video image 112 based on a video signal 111 that is a sports broadcast, the viewer may cheer louder than in other scenes, make a fist pump, or become excited and have a rise in body temperature during a highlight scene such as a score scene. Also, during a highlight scene such as a score scene, the viewer may stomp their feet or do other such things due to excitement. Therefore, the volume change detection unit 131 detects a change in volume when the viewer cheers. Also, the movement detection unit 132 detects a change in movement when the viewer makes a fist pump. Also, the vibration of the floor caused by the viewer's stomp or the like is transmitted to the installation stand of the display device 100, and the installation stand vibrates, causing the display device 100 to vibrate. As a result, for example, when a vibration meter 123 is installed on the display device 100, the vibration meter 123 measures the vibration of the display device 100. Alternatively, the vibration meter 123 may measure the vibration of the floor as the vibration generated by the viewer. The vibration change detection unit 133 detects a change in the vibration measured by the vibration meter. Additionally, the body temperature change detection unit 134 detects changes in body temperature when a viewer becomes excited.

As described above, the display device 100 identifies highlight scenes based on changes in the viewers' behavior, and can therefore identify scenes in which the viewers actually became excited as highlight scenes.

(Variation 1)
As a first modified example of the display device 100 according to the present embodiment, the acquisition unit 105 may acquire, as a change in the viewer, that the viewer of the display device 100 has operated the operation unit 109. For example, while watching a moving image, the viewer presses a predetermined button on the operation unit 109, such as a remote control, at a scene that the viewer thinks is a highlight scene. The acquisition unit 105 acquires a signal indicating that the predetermined button has been pressed. In this case, the identification unit 106 may identify the scene where the predetermined button has been pressed as a highlight scene. This allows the display device 100 to identify the scene designated by the viewer as a highlight scene.

(Variation 2)
As a second modified example of the display device 100 according to the present embodiment, when a highlight scene is identified, the identification unit 106 may attach a tag to at least one of a portion corresponding to the beginning and a portion corresponding to the end of the scene identified as the highlight scene, among the video signals 113 stored in the storage unit 104. In this way, when a plurality of video signals 113 corresponding to different highlight scenes are stored in the storage unit 104, the display device 100 can distinguish each highlight scene based on the tag.

(Variation 3)
As a third modified example of the display device 100 according to the present embodiment, the identification unit 106 may notify the viewer that highlight detection is in progress from the start of highlight scene identification until it is determined that the highlight scene has ended. For example, the identification unit 106 outputs a message "highlight detection in progress" to the display unit 103 from the start of highlight scene identification until it is determined that the highlight scene has ended, thereby causing the display unit 103 to display the message. Alternatively, the identification unit 106 may output a symbol or the like indicating that highlight detection is in progress to the display unit 103 from the start of highlight scene identification until it is determined that the highlight scene has ended. For example, the identification unit 106 may cause the display unit 103 to flash a symbol or the like indicating that highlight detection is in progress.

(Variation 4)
For example, there is a risk that the cheers of the viewers during the highlight scene will drown out the sound output from the display device 100. Therefore, as a fourth modified example of the display device 100 according to the present embodiment, the identification unit 106 may increase the volume of the voice, music, sound effects, and the like included in the video signal 111 from the start of identification of the highlight scene until it is determined that the highlight scene has ended. This allows the display device 100 to allow the viewer to enjoy the highlight scene without the sound output from the display device 100 being drowned out by the cheers of the viewers during the highlight scene.

(Variation 5)
As a fifth modification of the display device 100 according to the present embodiment, the determination unit 106 may count the number of times a highlight scene has been identified for the same video signal 111. Furthermore, the determination unit 106 may determine the excitement level of the viewer based on a physical quantity generated by a viewer who satisfies a highlight condition. For example, the determination unit 106 determines that the greater the physical quantity generated by a viewer who satisfies a highlight condition, the higher the excitement level of the viewer. Then, the determination unit 106 stores the number of times a highlight scene has been identified and the excitement level of the viewer in the storage unit 104.

Second Embodiment
The second embodiment will be described with reference to Figures 4 and 5. In the drawings, the same or equivalent elements are given the same reference numerals, and duplicated explanations are omitted. Configurations and processes having substantially the same functions as the other embodiments are referred to by the same reference numerals, and explanations are omitted, and differences from the other embodiments are described.

Fig. 4 is a diagram showing an example of the configuration of a display device 400 according to this embodiment. The display device 400 shown in Fig. 4 differs from the display device 100 shown in Fig. 1 in that the display device 400 shown in Fig. 4 includes an image synthesis unit 401 instead of the image switching unit 108.

The video synthesis unit 401 synthesizes the video 112 with the video 114 so that the video 112 based on the received video signal 111 and the video 114 based on the video signal 113 corresponding to the highlight scene are displayed on the display unit 103.

Next, the operation of the display device 400 will be described with reference to FIG. 5. FIG. 5 is a flowchart showing an example of the operation of the display device 400. When the content input unit 101 receives a video signal 111, the display device 400 starts the process of step S201 illustrated in FIG. 2. The display device 400 then executes the processes of steps S202 to S206 illustrated in FIG. 2. When the playback unit 107 receives an instruction to play back a highlight scene by a viewer's operation in step S206, the display device 400 executes the process of step S207 illustrated in FIG. 2, and then executes the process of S501 illustrated in FIG. 5.

In step S501, the video synthesis unit 401 synthesizes the video 112 based on the video signal 111 received in step S201 with the video 114 based on the video signal 113 corresponding to the highlight scene played in step S207. For example, the video synthesis unit 401 synthesizes the video 114 based on the video signal 113 corresponding to the highlight scene with the video 112 so that the video 114 based on the video signal 113 corresponding to the highlight scene is displayed in PinP (Picture in Picture) format, superimposed on a partial area of the video 112.

In step S502, the video synthesis unit 401 outputs the moving image synthesized in step S501 to the display unit 103, thereby causing the display unit 103 to display the moving image 114 based on the video signal 113 corresponding to the highlight scene and the moving image 112 based on the received video signal 111. In this way, the display device 400 allows the viewer to view the highlight scene together with the moving image 112 based on the received video signal 111.

For example, if the number of viewers increases while viewing video 112 based on received video signal 111, the viewers perform an operation to play back highlight scenes. In this case, display device 400 allows the viewers to view highlight scenes while also viewing video 112 based on video signal 111. This allows display device 400 to allow new viewers to view highlight scenes, allowing viewers who had been viewing video 112 to share in the excitement of the video.

When playback of video 114 based on video signal 113 corresponding to the highlight scene is completed in step S209, in step S210, video synthesis unit 401 ends synthesis of video 114 with video 112, and outputs video 112 based on video signal 111 to display unit 103.

As a result, the display device 400 can allow the viewer to watch the highlight scene while also allowing the viewer to watch the rest of the video.

Third Embodiment
The third embodiment will be described with reference to Figures 6 and 7. In the drawings, the same or equivalent elements are given the same reference numerals, and duplicated explanations are omitted. Configurations and processes having substantially the same functions as the other embodiments are referred to by the same reference numerals, and explanations are omitted, and differences from the other embodiments are described.

Fig. 6 is a diagram showing an example of the configuration of a display device 600 according to this embodiment. The difference between the display device 600 shown in Fig. 6 and the display device 400 shown in Fig. 4 is that the display device 600 shown in Fig. 6 includes a viewer number sensor 601.

The viewer number sensor 601 detects the number of viewers watching the video displayed on the display unit 103. The viewer number sensor 601 includes a face detection unit 611 and a heat source detection unit 612. The face detection unit 611 detects face areas from the captured image generated by the image sensor 122, and calculates the number of detected face areas. The number of face areas is the number of face areas included in the captured image. The heat source detection unit 612 calculates the number of heat sources based on temperature information acquired by the infrared camera 124. The number of heat sources indicates the number of viewer areas.

After the viewer count sensor 601 detects an increase in the number of viewers, the playback unit 107 according to this embodiment plays back video based on the video signal stored in the storage unit, including highlight scenes. Specifically, when the number of face regions detected by the face detection unit 611 increases, the playback unit 107 according to this embodiment plays back video 114 based on the video signal 113 corresponding to the highlight scene. Furthermore, when the number of heat sources detected by the heat source detection unit 612 increases, the playback unit 107 according to this embodiment plays back video 114 based on the video signal 113 corresponding to the highlight scene. In other words, when the number of viewers increases, the playback unit 107 according to this embodiment plays back highlight scenes.

Next, the operation of the display device 600 will be described with reference to FIG. 7. When the content input unit 101 of the display device 600 receives a video signal 111, the display device 600 starts the process of step S201 illustrated in FIG. 2. The display device 600 then executes the processes of steps S202 to S205 illustrated in FIG. 2. After the identification unit 106 stores the video signal 113 corresponding to the identified highlight scene from the video signal 111 in the storage unit 104 in step S205, the display device 600 executes the process of step S701 illustrated in FIG. 7.

In step S701, the face detection unit 611 detects face areas from the captured image generated by the image sensor 122, and calculates the number of face areas. For example, the face detection unit 611 detects feature points such as the eyes, nose, and mouth from the captured image to detect face areas, and calculates the number of detected face areas.

In step S702, the heat source detection unit 612 calculates the number of heat sources based on the temperature information from the infrared camera. Specifically, the heat source detection unit 612 detects areas of the temperature information acquired by the infrared camera 124 that are equal to or larger than a predetermined size and have a temperature within a predetermined range as heat sources, and calculates the number of heat sources. The predetermined temperature range is a temperature that corresponds to human body temperature, for example, 35°C to 40°C.

In step S703, the playback unit 107 determines whether at least one of the number of face regions calculated in step S701 and the number of heat sources calculated in step S702 has increased. In other words, in step S703, the playback unit 107 determines whether the number of viewers has increased.

If it is not determined in step S703 that the number of face regions or the number of heat sources has increased, the display device 600 transitions to step S201, as exemplified in FIG. 2. In other words, if it is not determined that the number of viewers has increased, the display device 600 returns the process to step S201 and continues playing the video 112 based on the video signal 111. On the other hand, if the playback unit 107 determines in step S703 that at least one of the number of face regions and the number of heat sources has increased, in step S704, the video synthesis unit 401 synthesizes the video 112 based on the video signal 111 with the video 114 based on the video signal 113 corresponding to the highlight scene. In step S705, the video synthesis unit 401 outputs the synthesized video to the display unit 103, thereby causing the display unit 103 to display the video 114 based on the video signal 113 corresponding to the highlight scene and the video 112 based on the received video signal 111. The processing in steps S704 and S705 is similar to the processing in steps S501 and S502 illustrated in FIG. 5, so a detailed description will be omitted.

As described above, the display device 600 determines whether the number of viewers has increased based on the captured image generated by the imaging element 122 or the temperature information acquired by the infrared camera 124. Then, when the number of viewers has increased, the display device 600 allows the viewers to view the video 114 based on the video signal 113 corresponding to the highlight scene. In other words, when the number of viewers increases, the display device 600 displays the highlight scene, so that new viewers can view the highlight scene without any operation by the viewer. Therefore, in the display device 600, viewers who have been viewing the video 112 until now can continue viewing, while new viewers can share the excitement of the video 112 as they start viewing the video 112.

(Fourth embodiment)
The fourth embodiment will be described with reference to Figures 8 and 9. In the drawings, the same or equivalent elements are given the same reference numerals, and duplicated explanations are omitted. Configurations and processes having substantially the same functions as the other embodiments are referred to by the same reference numerals, and explanations are omitted, and differences from the other embodiments are described.

Fig. 8 is a diagram showing an example of the configuration of a display device 800 according to this embodiment. The difference between the display device 800 illustrated in Fig. 8 and the display device 100 illustrated in Fig. 1 is that the display device 800 illustrated in Fig. 8 includes an image change detection unit 801 and an image volume detection unit 802.

The image change detection unit 801 detects a specific change in the video 112 based on a change in the movement of the image of an object included in the video 112. A specific change in the video 112 is the amount of change in pixel values within an area of a predetermined size in each frame included in the video 112. The predetermined size of each frame may be, for example, a size corresponding to a ratio to the number of pixels in the video 112.

The video volume detection unit 802 detects the amount of change in the sound of the video 112 based on the amount of change in the sound volume contained in the video 112. The amount of change in the sound of the video 112 is the amount of change in the sound volume within a predetermined period of time contained in the video 112.

The identification unit 106 according to this embodiment determines whether or not at least one of the specific change in the video 112 detected by the video change detection unit 801 and the amount of audio change in the video 112 detected by the video volume detection unit 802 satisfies a predetermined video change highlight condition. For example, the video change highlight condition is that at least one of the specific change in the video 112 and the amount of audio change in the video 112 is equal to or greater than a predetermined threshold.

Next, the operation of the display device 800 will be described with reference to FIG. 9. When the content input unit 101 receives a video signal 111, the display device 800 starts the process of step S201 illustrated in FIG. 2. Then, when the video processing unit 102 plays back the video image 112 based on the received video signal 111 in step S201 illustrated in FIG. 2, the display device 800 starts the process of step S901 illustrated in FIG. 9.

In step S901, the image change detection unit 801 detects a specific change in the video 112 based on a change in the image contained in the video signal 111. For example, if the video signal 111 is a video signal of a sports broadcast, a change in the video scene, such as a score display being displayed in a larger size, may be seen in a scoring scene. In that case, a specific change is seen in the image contained in the video signal 111. As a result, if the video signal 111 is a video signal of a sports broadcast, a specific change is seen in the video 112 in a scoring scene.

For example, if the video signal 111 is a video signal for a live sports broadcast, in a replay scene of a scoring scene, the camera angle changes multiple times, and a specific change such as a change in the image on the entire screen is observed. In this case, in a replay scene of a scoring scene, a specific change is observed in the video image 112.

In step S902, the video volume detection unit 802 detects the amount of change in the sound of the video 112 based on the amount of change in the sound volume contained in the video signal 111. For example, if the video signal 111 is a video signal of a sports broadcast, the volume of sounds such as cheers from the venue and the announcer's commentary will be louder in a scoring scene than in other scenes. Therefore, if the video signal 111 is a video signal of a sports broadcast, the amount of change in the sound of the video 112 will be louder in a scoring scene than in other scenes.

In step S903, the acquisition unit 105 acquires a physical quantity resulting from the viewer that indicates a change in the viewer. The process of step S903 is similar to the process of step S202 illustrated in FIG. 2, and therefore a detailed description thereof will be omitted.

In step S904, the identification unit 106 determines whether or not at least one of the specific change in the video 112 detected in step S901 and the amount of audio change in the video 112 detected in step S902 satisfies a predetermined video change highlight condition.

If at least one of the specific change in the video 112 and the amount of audio change in the video 112 does not satisfy the predetermined video change highlight condition, the display device 800 transitions the process to step S201 illustrated in FIG. 2. In other words, if at least one of the specific change in the video 112 and the amount of audio change in the video 112 does not satisfy the predetermined video change highlight condition, the display device 800 returns the process to step S201 and continues playing the video 112 based on the video signal 111. On the other hand, if at least one of the specific change in the video 112 and the amount of audio change in the video 112 satisfies the predetermined video change highlight condition, the display device 800 transitions the process to step S905.

In step S905, the identification unit 106 determines whether the physical quantity generated by the viewer acquired in step S903 satisfies a predetermined highlight condition. The process of step S905 is similar to the process of step S202 illustrated in FIG. 2, and therefore a detailed description thereof will be omitted.

In step S906, the identification unit 106 identifies a scene in which the predetermined image change highlight condition is satisfied and the predetermined highlight condition is satisfied as a highlight scene. Then, the display device 800 proceeds to step S205 illustrated in FIG. 2.

As described above, the display device 800 identifies the content of the video 112 and the scenes that excited the viewer as highlight scenes. This allows the display device 800 to identify more appropriate scenes in the video signal 111 as highlight scenes.

Fifth Embodiment
The fifth embodiment will be described with reference to Figures 10 and 11. In the drawings, the same or equivalent elements are given the same reference numerals, and duplicated explanations are omitted. Configurations and processes having substantially the same functions as the other embodiments are referred to by the same reference numerals, and explanations are omitted, and differences from the other embodiments are described.

Fig. 10 is a diagram showing an example of the configuration of a display device 1000 according to this embodiment. The difference between the display device 1000 illustrated in Fig. 10 and the display device 100 illustrated in Fig. 1 is that the display device 1000 illustrated in Fig. 10 includes a video storage unit 1001 in the identification unit 106.

The video storage unit 1001 repeatedly stores and updates the video signal 111 at a predetermined time interval.

The identification unit 106 according to the present embodiment stores in the storage unit 104 a video signal 1002 corresponding to a scene from a scene prior to the scene identified as a highlight scene to the end of the identified highlight scene. Specifically, the identification unit 106 according to the present embodiment stores in the storage unit 104 a video signal 1002 consisting of a video signal for one interval of a predetermined time going back from the identified highlight scene stored in the video storage unit 1001 and a video signal corresponding to the identified highlight scene. The playback unit 107 according to the present embodiment plays a video 1003 based on the video signal 1002.

Next, the operation of the display device 1000 will be described with reference to FIG. 11. When the content input unit 101 receives a video signal 111, the display device 1000 starts the process of step S201 illustrated in FIG. 11.

When the video processing unit 102 plays back the video 112 based on the video signal 111 received by the content input unit 101 in step S201, the identification unit 106 in step S1101 stores in the video storage unit 1001 a video signal that goes back a predetermined time from the video signal 111 corresponding to the played video 112. The display device 1000 then transitions to step S202. The processes in steps S202 to S204 illustrated in FIG. 11 are similar to steps S202 to S204 illustrated in FIG. 2, and therefore will not be described in detail.

If the identification unit 106 identifies a highlight scene in step S204, in step S1102, the identification unit 106 stores in the storage unit 104 the video signal stored in the video storage unit 1001 in step S1101 and the video signal corresponding to the scene up to the end of the highlight scene identified in step S204. Then, the display device 1000 transitions to the process in step S206.

If the playback unit 107 receives an instruction to play back a highlight scene by a viewer's operation in step S206, then in step S1103, the playback unit 107 plays back the video image 1003 based on the video signal 1002 stored in the storage unit 104.

The timing when viewers get excited and excited is after an event that could be a highlight scene occurs. For example, in a soccer broadcast, viewers get excited and excited when they see a goal being scored. Similarly, in a baseball broadcast, viewers get excited and excited when they see a home run being hit. Therefore, if the identification unit 106 identifies the scene being played as a highlight scene after the physical quantity generated by the viewer satisfies a predetermined highlight condition, there is a risk that the scene in which an event that could be a highlight scene occurs will not be identified as a highlight scene. However, the playback unit 107 can play back video 1003 based on a video signal corresponding to a scene from before the identified highlight scene to the end of the highlight scene, thereby playing back video 1003 including a scene in which an event that could be a highlight scene occurs.

In step S1104, the video switching unit 108 switches between the video 112 based on the received video signal 111 and the video 1003 played in step S1103, and outputs the video to the display unit 103. When the playback of the video 1003 ends in step S209, the display device 1000 executes the process of step S210. The process of step S210 is similar to step S210 illustrated in FIG. 2, and therefore a detailed description thereof will be omitted.

As described above, the display device 1000 displays the video 1003 on the display unit 103 from a scene prior to the identified highlight scene to the end of the highlight scene. Therefore, the display device 1000 can allow the viewer to view the events leading up to the highlight scene and the scene in which an event that could become a highlight scene occurred.

The control blocks of the display device 100, the display device 400, the display device 600, the display device 800, and the display device 1000 (particularly the video processing unit 102, the acquisition unit 105, the identification unit 106, the playback unit 107, the video switching unit 108, the video synthesis unit 401, the video change detection unit 801, and the video volume detection unit 802) can be realized by a logic circuit (hardware) formed in an integrated circuit (IC (Integrated Circuit) chip) or the like, or may be realized by software using a CPU (Central Processing Unit). In the latter case, the display device 100, the display device 400, the display device 600, the display device 800, and the display device 1000 are equipped with a CPU that executes the instructions of a program, which is software that realizes each function, a ROM or storage device (these are referred to as "recording media") in which the above program and various data are recorded so as to be readable by a computer (or CPU), and a RAM that deploys the above program. The object of the present invention is achieved by the computer (or CPU) reading and executing the above program from the recording medium. The recording medium may be a "non-transitory tangible medium," such as a tape, disk, card, semiconductor memory, programmable logic circuit, etc. The program may be supplied to the computer via any transmission medium capable of transmission.

This disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. The technical scope of the present invention also includes embodiments obtained by appropriately combining the technical means disclosed in different embodiments. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

100 Display device, 101 Content input unit, 102 Video processing unit, 103 Display unit, 104 Memory unit, 105 Acquisition unit, 106 Identification unit, 107 Playback unit, 108 Video switching unit, 109 Operator, 111 Video signal, 112 Video image, 113 Video signal, 114 Video image, 121 Microphone, 122 Image sensor, 123 Vibration meter, 124 Infrared camera, 131 Volume change detection unit, 132 Detection unit, 133 Vibration change detection unit, 134 Body temperature change detection unit, 400 Display device, 401 Video synthesis unit, 500 Display device, 501 Video synthesis unit, 600 Display device, 601 Viewer number sensor, 611 Face detection unit, 612 Heat source detection unit, 800 Display device, 801 Video change detection unit, 802 Video volume detection unit, 1000 display device, 1001 video storage unit, 1002 video signal, 1003 video image

Claims (5)

a display unit that displays a moving image based on the received video signal;
an acquisition unit that acquires a change in a viewer who is viewing the moving image displayed on the display unit;
an identification unit that identifies a highlight scene of the video based on the change in viewers acquired by the acquisition unit;
A storage unit that stores the video signal;
a viewer number sensor for detecting the number of viewers viewing the video;
After the viewer number sensor detects an increase in the number of viewers, the video based on the received video signal is combined with the video based on the video signal stored in the storage unit, including the highlight scene, and the combined video is displayed on the display unit;
the acquisition unit is a sensor that detects a physical quantity generated by the viewer,
The display device wherein, when a physical quantity generated from the viewer acquired by the acquisition unit becomes equal to or greater than a predetermined first threshold and then becomes equal to or greater than a second threshold that exceeds the first threshold, the identification unit identifies a scene up to when the physical quantity becomes equal to or greater than the second threshold as the highlight scene, and determines that the highlight scene has ended if the physical quantity does not become equal to or greater than the second threshold within a predetermined time after the physical quantity becomes equal to or greater than the second threshold . The display device according to claim 1, wherein, when the highlight scene is identified by the identification unit, the storage unit stores a scene from an earlier scene than the scene identified as the highlight scene, and plays a video image based on the stored video signal. The display device according to claim 1, wherein, when the highlight scene is identified by the identification unit, a tag is added to at least one of a portion of the video signal stored in the storage unit that corresponds to the beginning and the end of the scene identified as the highlight scene. The display device according to claim 1 , wherein the sensor is at least one of a microphone that captures a sound generated by the viewer and an imaging element that captures an image of the viewer. The display device further includes an operator operated by a viewer of the display device,
4. The display device according to claim 1, wherein the acquisition section acquires, as the change in the viewer, that the viewer of the display device has operated the control.

Images