JP5226414B2 - Sign language interpreting system, broadcasting device, sign language interpreting device, broadcasting method, and program - Google Patents

Description

  The present invention relates to a sign language interpretation system that broadcasts broadcast program information and also broadcasts sign language video information indicating a sign language video corresponding to the broadcast program information.

Conventionally, a broadcast program in which a sign language image is inserted may be broadcast in a broadcast station. However, such programs are limited, and the hearing impaired cannot obtain sign language images for any broadcast program. When sign language images are inserted for all broadcast programs, it is necessary for sign language interpreters to go to broadcasting stations, which increases the burden on sign language interpreters. Therefore, a system has been developed that provides a sign language video related to a broadcast program by performing sign language interpretation on a network (see, for example, Patent Document 1).
JP 2002-169988 A

  However, in the sign language interpretation system described in Patent Document 1, for example, when a hearing-impaired person views a broadcast program and a sign language video corresponding to the broadcast program, an apparatus that receives the broadcast program (for example, a television receiver). Etc.) and a device (for example, a personal computer) connected to a network for receiving sign language images must be prepared, which imposes complicated work on the hearing impaired. In particular, when watching a broadcast program at another person's house or watching a broadcast program outdoors, it is often very difficult to set up such an environment.

  Even if a device for receiving a broadcast program and a device for receiving a sign language image are installed side by side, it is very difficult to see the images of both devices at a glance, and the image of the broadcast program is being viewed. In some cases, sign language images cannot be viewed, and when viewing sign language images, it is often impossible to view broadcast program images.

  The present invention has been made in order to solve the above-described problems, and can perform sign language interpretation on a network, and a broadcast program and a sign language video corresponding to the broadcast program can be obtained by a single broadcast receiving apparatus. An object is to provide a sign language interpreting system and the like that can be received.

  In order to achieve the above object, a sign language interpreting system according to the present invention is a sign language interpreting system including a broadcasting device and a sign language interpreting device, and the broadcasting device transmits a sign language image transmitted from the sign language interpreting device. A sign language video information receiving unit that receives sign language video information that is information indicating a broadcast program information storage unit that stores broadcast program information that is information indicating a program to be broadcast, and the broadcast program information storage unit And a broadcast unit that broadcasts the sign language video information received by the sign language video information receiving unit, and the sign language interpreting device receives the broadcast program information broadcast by the broadcast device. Broadcast program information receiver, video / audio output unit for outputting video / audio of broadcast program information received by the broadcast program information receiver, and sign language corresponding to the video / audio output by the video / audio output unit A sign language video information receiving unit that receives sign language video information that is information indicating a video; and a sign language video information transmitting unit that transmits the sign language video information received by the sign language video information receiving unit to the broadcasting device. .

  With such a configuration, sign language interpretation can be performed on the network, and broadcast program information and sign language video information can be received by one broadcast receiving apparatus. Therefore, a hearing-impaired person can watch the video of both information by one apparatus, without preparing two apparatuses, the apparatus which receives broadcast program information, and the apparatus which receives sign language image information. become. Also, for example, when broadcast program information and sign language video information are each broadcast on one channel, it is possible to simultaneously receive broadcasts on two channels and simultaneously output video on the two channels. By using a possible general-purpose broadcast receiver, both broadcast program information and sign language video information can be viewed simultaneously.

  In the sign language interpretation system according to the present invention, the broadcast device stores a delay time information that is information indicating a delay time of the sign language video information with respect to the broadcast program information broadcast by the broadcast unit. An information storage unit may be further provided, and the broadcast unit may also broadcast the delay time information.

  With such a configuration, delay time information is also broadcast, so a broadcast receiving apparatus that receives broadcast program information and sign language video information uses the delay time information to synchronize and output broadcast program information and sign language video information. It is possible to improve the convenience of hearing impaired people who view broadcast program information and sign language video information.

In the sign language interpretation system according to the present invention, the broadcasting unit may broadcast the sign language video information including the delay time information.
With such a configuration, it becomes easy to know which sign language video information the delay time information corresponds to.

In the sign language interpretation system according to the present invention, the broadcasting device may further include a delay time information acquisition unit that acquires the delay time information and stores the delay time information in the delay time information storage unit.
With such a configuration, the delay time information can be automatically acquired. Further, for example, by periodically acquiring delay time information, it is possible to cope with a delay time that changes with time.

In the sign language interpreting system according to the present invention, the delay time information acquisition unit measures a communication time of information between the broadcasting device and the sign language interpreting device, and the communication time and a sign language interpreter stored in advance are stored. You may acquire the delay time information which shows the time which added the time gap time.
With such a configuration, it is possible to measure a communication time that is considered to change greatly, and it is possible to acquire more accurate delay time information according to the situation.

In the sign language interpreting system according to the present invention, the broadcast unit broadcasts broadcast program information including a time code, and the sign language video information transmission unit is the latest in the broadcast program information received by the broadcast program information reception unit. The sign language video information including the time code is transmitted, and the delay time information acquisition unit transmits the sign language video information including the time code from the time when the broadcast unit broadcasts the broadcast program information including a certain time code. The communication time, which is the time until the information receiving unit receives it, may be acquired, and delay time information indicating a time obtained by adding the communication time and a preliminarily held shift time during sign language interpretation may be acquired. .
With such a configuration, it is possible to acquire delay time information using a time code.

In the sign language interpretation system according to the present invention, the delay time information acquisition unit indicates the sign language video information received by the sign language video information reception unit from the start time of the sound included in the broadcast program information broadcast by the broadcasting unit. You may acquire the delay time information which shows the time to the start time of the sign language of an image | video.
With such a configuration, it is possible to acquire more accurate delay time information considering a time lag that occurs when a sign language interpreter performs sign language interpretation.

The sign language interpretation system according to the present invention further comprises a broadcast receiving device, the broadcast receiving device receiving broadcast program information, sign language video information and delay time information broadcast by the broadcast device, and the broadcast An output unit that outputs the video of the sign language video information received by the receiving unit and the video of the broadcast program information delayed by the delay time indicated by the delay time information received by the broadcast receiving unit may be provided.
With such a configuration, the broadcast receiving apparatus can output the broadcast program information synchronized with the delay time information and the sign language video information.

  According to the sign language interpretation system and the like according to the present invention, sign language interpretation can be performed on a network, and a broadcast program and a sign language video corresponding to the broadcast program can be received by one broadcast receiving apparatus. can do.

  Hereinafter, a sign language interpreting system according to the present invention will be described using embodiments. In the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and repetitive description may be omitted.

(Embodiment 1)
A sign language interpretation system according to Embodiment 1 of the present invention will be described with reference to the drawings. The sign language interpretation system according to the present embodiment can perform sign language interpretation on a network and broadcasts broadcast program information and sign language video information that is a sign language video corresponding to the broadcast program information from a broadcasting device. It is.

  FIG. 1 is a block diagram showing a configuration of a sign language interpretation system according to this embodiment. The sign language interpreting system according to the present embodiment includes a broadcasting device 1, a sign language interpreting device 2, and a broadcast receiving device 3. The sign language interpreting device 2 and the broadcast receiving device 3 can receive information broadcast by the broadcasting device 1. Further, it is assumed that the sign language interpreting apparatus 2 and the broadcasting apparatus 1 are connected by a wired or wireless communication line 500. The communication line 500 is, for example, the Internet, an intranet, a public telephone line network, or the like. Although FIG. 1 shows a case where there is one sign language interpreting device 2 and one broadcast receiving device 3, the sign language interpreting system may include two or more sign language interpreting devices 2, and 2 Needless to say, the broadcast receiving apparatus 3 may be provided. For example, by using two or more sign language interpreting apparatuses 2, sign language interpretation may be performed alternately by two or more sign language interpreters.

  FIG. 2 is a block diagram showing a configuration of the broadcast apparatus 1 according to the present embodiment. In FIG. 2, the broadcast apparatus 1 according to the present embodiment includes a broadcast program information storage unit 11, a broadcast unit 12, a sign language video information reception unit 13, a delay time information acquisition unit 14, and a delay time information storage unit 15. Is provided.

  The broadcast program information storage unit 11 stores broadcast program information. Broadcast program information is information indicating a program broadcast by the broadcast device 1. The broadcast program information may be, for example, drama broadcast program information, news broadcast program information, documentary broadcast program information, sports broadcast program information, etc., and the genre thereof is not limited. Moreover, the data format of broadcast program information is not ask | required. For example, the broadcast program information may be analog data or digital data. In the latter case, the broadcast program information format may be, for example, MPEG (Moving Picture Experts Group), AVI (Audio Video Interleave), or any other format. Further, when the broadcast program information is compressed, the compression format is not limited.

  The process in which broadcast program information is memorize | stored in the broadcast program information storage part 11 is not ask | required. For example, broadcast program information may be stored in the broadcast program information storage unit 11 via a recording medium, and broadcast program information transmitted via a communication line or the like is stored in the broadcast program information storage unit 11. Alternatively, broadcast program information input from a camera or the like may be stored in the broadcast program information storage unit 11. Storage in the broadcast program information storage unit 11 may be temporary storage in a RAM or the like, or may be long-term storage. The broadcast program information storage unit 11 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The broadcast unit 12 broadcasts broadcast program information stored in the broadcast program information storage unit 11. This broadcast is similar to a so-called television broadcast. The broadcast unit 12 also broadcasts sign language video information received by a sign language video information receiving unit 13 described later. The sign language video information will be described later. The broadcast unit 12 may also broadcast the delay time information stored in the delay time information storage unit 15 described later. The delay time information will be described later. When the delay time information is also broadcast, the broadcast unit 12 may broadcast the sign language video information including the delay time information. Including the delay time information in the sign language video information means that the sign language video information and the delay time information are transmitted through one channel. Therefore, for example, the delay time information may be included in the user-defined area of the sign language video information, and the delay time information may be multiplexed in the vertical blanking interval as in the closed caption. The broadcast unit 12 may broadcast the delay time information using BML (Broadcast Markup Language). The broadcast by the broadcast unit 12 may be, for example, a terrestrial broadcast, a satellite broadcast, or a wired broadcast via a cable. In addition, the broadcast unit 12 may appropriately read and broadcast the broadcast program information according to information indicating a schedule of broadcast program information to be broadcast.

  The broadcasting unit 12 may or may not include a wired or wireless broadcasting device for performing broadcasting. The broadcast unit 12 may be realized by hardware, or may be realized by software such as a driver that drives a broadcast device.

  The sign language image information receiving unit 13 receives sign language image information transmitted from the sign language interpreting apparatus 2 described later via the communication line 500. The sign language video information is information indicating a sign language image, and is information indicating a sign language image corresponding to broadcast program information broadcast by the broadcast unit 12 as described later. Details of the sign language video information will be described later.

  The sign language video information receiving unit 13 may or may not include a wired or wireless receiving device (for example, a modem or a network card) for receiving. In addition, the sign language video information receiving unit 13 may be realized by hardware, or may be realized by software such as a driver that drives a receiving device.

  The delay time information acquisition unit 14 acquires delay time information. The delay time information acquisition unit 14 accumulates the acquired delay time information in the delay time information storage unit 15. The delay time information is information indicating the time that the sign language video information is delayed with respect to the broadcast program information broadcast by the broadcast unit 12. Details thereof and a method by which the delay time information acquisition unit 14 acquires the delay time information will be described later. Obtaining the delay time information may be, for example, accumulating the delay time information in a recording medium. The storage medium as the storage destination may be the delay time information storage unit 15. In addition, the delay time information acquisition unit 14 may acquire delay time information for each sign language interpreting device 2, for example, may acquire delay time information for each broadcast program information, or at predetermined time intervals. Delay time information may be acquired, or delay time information may be acquired at other timings.

  The delay time information storage unit 15 stores the delay time information accumulated by the delay time information acquisition unit 14. The storage in the delay time information storage unit 15 may be temporary storage in a RAM or the like, or may be long-term storage. The delay time information storage unit 15 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The broadcast program information storage unit 11 and the delay time information storage unit 15 may be realized by the same recording medium, or may be realized by separate recording media. In the former case, the area storing the broadcast program information is the broadcast program information storage unit 11, and the area storing the delay time information is the delay time information storage unit 15.

  FIG. 3 is a block diagram showing a configuration of the sign language interpreting apparatus 2 according to the present embodiment. In FIG. 3, the sign language interpreting apparatus 2 according to the present embodiment includes a broadcast program information receiving unit 21, a video / audio output unit 22, a sign language video information receiving unit 23, and a sign language video information transmitting unit 24.

  The broadcast program information receiving unit 21 receives broadcast program information broadcast by the broadcast device 1. The broadcast program information receiving unit 21 may or may not include a wired or wireless receiving device for receiving. The broadcast program information receiving unit 21 may be realized by hardware, or may be realized by software such as a driver that drives a receiving device.

  The video / audio output unit 22 outputs the video / audio of the broadcast program information received by the broadcast program information receiving unit 21. Here, this output may be, for example, display of broadcast program information video on a display device (for example, a CRT or a liquid crystal display) and audio output of broadcast program information to a speaker. Alternatively, this output may be, for example, transmission of video / audio via a communication line to a predetermined device, accumulation in a recording medium, or delivery to another component. However, even in these cases, the video is ultimately displayed and the audio is output as audio. Note that the video / audio output unit 22 may or may not include an output device (for example, a display device or a speaker). The video / audio output unit 22 may be realized by hardware, or may be realized by software such as a driver for driving these devices.

  Here, the video / audio output by the video / audio output unit 22 is viewed by a sign language interpreter. Then, the sign language interpreter performs sign language interpretation of the broadcast program information. That is, the sign language interpreter performs sign language corresponding to the video and audio of the broadcast program information so that the sign language is photographed by the camera.

  The sign language video information accepting unit 23 accepts sign language video information which is information indicating a sign language video corresponding to the video and audio output by the video and audio output unit 22. The sign language video information is information indicating a sign language video in which the broadcast program information is interpreted by the sign language interpreter. Moreover, the data format of sign language video information is not ask | required similarly to broadcast program information. For example, the sign language video information may be analog data or digital data. Further, when the sign language video information is compressed, the compression format is not limited.

  The sign language video information receiving unit 23 may receive, for example, sign language video information input from a camera, may receive sign language video information transmitted via a wired or wireless communication line, and a predetermined recording medium. Sign language video information read from (for example, an optical disk, a magnetic disk, a semiconductor memory, etc.) may be received. Note that the sign language video information receiving unit 23 may or may not include a device (for example, a modem or a network card) for receiving. The sign language video information receiving unit 23 may be realized by hardware, or may be realized by software such as a driver that drives a predetermined device.

  The sign language video information transmission unit 24 transmits the sign language video information received by the sign language video information reception unit 23 to the broadcasting device 1 via the communication line 500. Note that the sign language video information transmission unit 24 may directly transmit to the broadcasting apparatus 1 or may indirectly transmit via another server or the like. In addition, the sign language video information transmission unit 24 may or may not include a transmission device (for example, a modem or a network card) for performing transmission. In addition, the sign language video information transmission unit 24 may be realized by hardware, or may be realized by software such as a driver that drives a transmission device.

  FIG. 4 is a block diagram showing a configuration of the broadcast receiving apparatus 3 according to the present embodiment. In FIG. 4, the broadcast receiving apparatus 3 according to the present embodiment includes a broadcast receiving unit 31, a broadcast program information storage unit 32, a broadcast program information storage unit 33, and an output unit 34. The broadcast receiving device 3 may be, for example, an STB (Set Top Box) connected to a television receiver or the like, or may not be so.

  The broadcast receiving unit 31 receives broadcast program information, sign language video information, and delay time information broadcast by the broadcast device 1. As described above, the broadcast program information and sign language video information received by the broadcast receiving unit 31 are generally transmitted through different channels. Further, as described above, the delay time information received by the broadcast receiving unit 31 may or may not be included in the sign language video information. In the former case, the broadcast receiver 31 has received the delay time information as a result of receiving the sign language video information. Further, in the broadcast receiving device 3, for example, when “sign language ON” is set, the broadcast receiving unit 31 receives a broadcast program information channel, and the sign language video corresponding to the broadcast program information is received. Information channels may also be automatically received. The channel of sign language video information corresponding to the channel of the broadcast program information may be determined by a rule (for example, a channel obtained by adding 1 to the channel of the broadcast program information indicates the sign language video information corresponding to the broadcast program information). It may be a rule such as a channel), or information that associates a channel of broadcast program information with a channel of sign language video information may be set.

  The broadcast receiving unit 31 may or may not include a wired or wireless receiving device for receiving. The broadcast receiving unit 31 may be realized by hardware, or may be realized by software such as a driver that drives a receiving device.

  The broadcast program information accumulation unit 32 accumulates the broadcast program information received by the broadcast reception unit 31 in the broadcast program information storage unit 33. This accumulation of broadcast program information is performed in order to delay the broadcast program information by the delay time indicated by the delay time information. That is, the broadcast program information is temporarily buffered in the broadcast receiving device 3.

  The broadcast program information storage unit 33 stores broadcast program information. This broadcast program information is stored by the broadcast program information storage unit 32 as described above. The storage in the broadcast program information storage unit 33 is generally temporary storage (buffering) in a RAM or the like, but may be long-term storage. The broadcast program information storage unit 33 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.). The storage of broadcast program information using the broadcast program information storage unit 33 may be buffering using a ring buffer, for example.

  The output unit 34 outputs the video of sign language video information received by the broadcast receiver 31 and the video of broadcast program information delayed by the delay time indicated by the delay time information received by the broadcast receiver 31. The output unit 34 outputs, for example, the sign language video information received by the broadcast receiving unit 31 as it is, and broadcast program information received by the broadcast receiving unit 31 and stored in the broadcast program information storage unit 33. Is output after being delayed by the delay time indicated by the delay time information. By delaying the output of the broadcast program information in this way, the time lag between the broadcast program information and the sign language video information can be absorbed and the broadcast program information and the sign language video information can be synchronized. The output unit 34 may output the sign language video information received by the broadcast receiving unit 31 with a delay. In that case, the broadcast program information received by the broadcast receiving unit 31 and stored in the broadcast program information storage unit 33 for the same time as the delay may be further delayed and output. . As described above, if the broadcast program information is output after being relatively delayed by the delay time indicated by the delay time information with respect to the sign language video information, the delay method is not limited.

  When the sign language video information is also delayed, the broadcast receiving apparatus 3 includes a sign language video information storage unit (not shown) for storing sign language video information and a sign language video information storage unit (not shown) for storing sign language video information. Further, it may be provided. The broadcast receiving apparatus 3 further includes a delay time information storage unit (not shown) that stores the delay time information received by the broadcast reception unit 31 and a delay time information storage unit (not shown) that stores the delay time information. The unit 34 may synchronize the broadcast program information and the sign language video information using the delay time information stored in the delay time information storage unit.

  Here, this output is, for example, a display of a video of broadcast program information and a video of sign language video information on a display device (for example, a CRT or a liquid crystal display) and an audio output of the broadcast program information to a speaker. There may be. Alternatively, this output may be, for example, transmission of video / audio via a communication line to a predetermined device, accumulation in a recording medium, or delivery to another component. However, even in these cases, the video is ultimately displayed and the audio is output as audio. The output unit 34 may or may not include an output device (for example, a display device or a speaker). The output unit 34 may be realized by hardware, or may be realized by software such as a driver that drives these devices. Further, when only the hearing impaired person sees the output of the output unit 34, the sound output may not be performed.

  In addition, when broadcast program information and sign language video information are displayed, they may be output side by side, vertically, diagonally, etc. within one screen, and the other screen may be part of one screen. May be output in a form in which broadcast program information and sign language video information are displayed.

  Needless to say, in the broadcasting apparatus 1, the sign language interpreting apparatus 2, and the broadcast receiving apparatus 3, information may be digitized or analogized as appropriate. Therefore, in the broadcasting device 1, the sign language interpreting device 2, and the broadcast receiving device 3, there are DACs (Digital Analog Converters) that convert information into analog and ADCs (Analog Digital Converters) that digitize information in necessary places. May be.

  Here, acquisition of the delay time information by the delay time information acquisition unit 14 of the broadcasting device 1 will be described. The delay time indicated by the delay time information is a delay time related to communication (hereinafter, sometimes referred to as “communication delay time”) and a shift time (the time during which the content of the broadcast program information and the operation of the sign language are shifted during sign language interpretation ( Hereinafter, it may be referred to as “sign language delay time”). The communication delay time may be, for example, the time taken when sign language video information is transmitted from the sign language interpreting apparatus 2 to the broadcasting apparatus 1, or broadcast program information is broadcast from the broadcasting apparatus 1 to the sign language interpreting apparatus 2. It may be a time obtained by adding the time required for the transmission of the sign language video information from the sign language interpretation device 2 to the broadcasting device 1. The sign language delay time is considered to be a fixed time even if it depends on the sign language interpreter. On the other hand, it is considered that the communication delay time may change to some extent depending on the degree of congestion of the communication line 500, the position of the sign language interpreter 2 on the network, and the like. Therefore, the delay time information acquisition unit 14 may measure the communication delay time, and may acquire delay time information indicating a time obtained by adding the measured communication delay time and a pre-stored sign language delay time. Further, the delay time information acquisition unit 14 may acquire a delay time itself that is a time obtained by adding the communication delay time and the sign language delay time. In this case, it is possible to acquire delay time information that also considers the sign language delay time. Hereinafter, each of these cases will be described in more detail.

[When measuring communication time]
The delay time information acquisition unit 14 measures a communication delay time which is a communication time of information between the broadcasting device 1 and the sign language interpreting device 2, and the communication delay time and a deviation time between the sign language interpretation held in advance. It is also possible to acquire delay time information indicating the time obtained by adding the interpretation delay time. The interpretation delay time held in advance may be held in a recording medium (not shown) by the delay time information acquisition unit 14, or may be held in another component or another device. In the latter case, the delay time information acquisition unit 14 receives the sign language delay time from another component or another device and uses it for acquiring the delay time information.

  In this case, as a method for the delay time information acquisition unit 14 to acquire the communication delay time, for example, ping may be used. That is, the delay time information acquisition unit 14 can transmit the destination of the ping packet as the sign language interpreter 2 and measure the round-trip communication time. Therefore, the communication delay time can be acquired by halving the round trip communication time measured by the ping. In general, in the ping process, a plurality of packet transmissions are performed. Therefore, an average of the communication times of the plurality of round trips may be calculated, and half of the average value may be used as the communication delay time. In addition, because of the protocol, the round-trip communication time at the time of transmission of the first first packet may be slower than the other cases, so the round-trip communication time measured using the second and subsequent packets. May be used to calculate the communication delay time.

  Here, although the case where ping is used as the case of measuring the communication time has been described, this is an example, and it goes without saying that the communication time may be measured using other methods.

[When using time code]
In this case, it is assumed that broadcast program information including a time code is broadcast by the broadcast unit 12 of the broadcast device 1. In addition, the sign language video information transmitting unit 24 of the sign language interpreting apparatus 2 transmits sign language video information including the latest time code included in the broadcast program information received by the broadcast program information receiving unit 21 to the broadcast apparatus 1. Then, the delay time information acquisition unit 14 of the broadcasting apparatus 1 starts the sign language video information reception unit 13 including the time code from the point when the broadcast unit 12 broadcasts the broadcast program information including a certain time code. A communication delay time that is a time until the reception time point is acquired, and delay time information indicating a time obtained by adding the communication delay time and the interpretation delay time held in advance may be acquired. The interpretation delay time held in advance may be held on a recording medium (not shown), for example, as described above. By doing in this way, the time taken when the broadcast program information is broadcast from the broadcasting device 1 to the sign language interpreting device 2, and the time taken when the sign language video information is transmitted from the sign language interpreting device 2 to the broadcasting device 1. It becomes possible to measure the time obtained by adding.

  For example, the time when a broadcast program information frame to which “00 (hour): 00 (minute): 12 (second): 01 (frame)”, which is one time code, is broadcast from the broadcast unit 12 is given. Suppose that it was A. The delay time information acquisition unit 14 temporarily stores information associating the time code with the time when the frame of the time code was broadcast. When the frame is received by the broadcast program information receiving unit 21, the time code “00: 00: 12: 01” is passed to the sign language video information transmitting unit 24, and the sign language video information transmitting unit 24 receives the time code. The time code is assigned to the frame of sign language video information to be transmitted immediately after receiving the code. The sign language video information frame is received by the sign language video information reception unit 13, and the time code included in the sign language video information frame is passed to the delay time information acquisition unit 14. When the delay time information acquisition unit 14 receives the time code “00: 00: 12: 01”, the time between the time B when the time code is received and the time A held in association with the time code Calculate The calculated time becomes the communication delay time.

[When using voice start time and sign language start time]
The delay time information acquisition unit 14 calculates the time from the start time of the sound included in the broadcast program information broadcast by the broadcast unit 12 to the start time of the sign language of the video indicated by the sign language video information received by the sign language video information reception unit 13. The delay time information shown may be acquired. In this case, the delay time itself, which is a time obtained by adding the communication delay time and the sign language delay time, can be acquired.

  For example, when one piece of broadcast program information is started, the delay time information acquisition unit 14 acquires the time of the start time of the sound included in the broadcast program information. The start time of the voice is a voice (for example, an utterance of an announcer, narrator, actor, or the like) corresponding to an utterance for which a sign language interpreter performs a sign language interpretation. Therefore, it is not the start time of sounds that are not subject to sign language interpretation, such as music and sound effects. Note that a method for distinguishing between non-speech sounds and speech sounds is known, for example, in Japanese Patent Application Laid-Open No. 2007-264567, and a detailed description thereof is omitted. In addition, a speech recognition process is performed on the sound included in the broadcast program information, and if the speech is recognized as a significant word string included in the dictionary information, it is determined that the speech is speech. You may judge that. Further, when the sign language video information corresponding to the one broadcast program information is received by the sign language video information receiving unit 13, the delay time information acquisition unit 14 at the time point when the sign language is started in the video indicated by the sign language video information. Get the time. The time when sign language is started may be, for example, a time when a difference between adjacent frames is detected and the difference is equal to or greater than a threshold value. It is more preferable that the detection of the difference can be limited to a person area. As a method for specifying a person's region, a skin color detection method, a method for detecting a person's outline or face position by pattern matching, or the like can be used. When at least a part of the person is specified, for example, the area of the whole person may be specified by using the specified region of at least a part of the person and the technique of contour extraction. When the delay time information acquisition unit 14 acquires the time of the start time of the sound included in the broadcast program and the time of the start time of the sign language in the sign language video information, the delay time information acquisition unit 14 calculates a time interval between the two times. The information indicating the calculated time interval becomes the delay time information as it is.

  Although several methods for the delay time information acquisition unit 14 to acquire the delay time information have been briefly described, the method for acquiring the delay time information is not limited to these, and another method is used for delay. Needless to say, time information may be acquired. For example, the sign language interpreting device 2 measures the communication delay time between the sign language interpreting device 2 and the broadcasting device 1 using ping or the like, and the communication delay time or a delay obtained by adding the sign language delay time to the communication delay time The time information itself may be transmitted to the broadcasting device 1, and the delay time information acquisition unit 14 may receive the communication delay time or the delay time information itself. In the former case, the delay time information can be obtained by adding the interpretation delay time held in advance.

  Further, the acquisition of the delay time information may be repeatedly executed or may be executed only once. In the former case, the latest value of the delay time information acquired repeatedly may be broadcast by the broadcast unit 12. In the latter case, the delay time information acquired once is repeatedly used by the broadcast receiving device 3. In the latter case, the delay time information acquired once may be broadcast repeatedly by the broadcast unit 12, or may be broadcast only once.

  Next, the operation of each device constituting the sign language interpretation system according to the present embodiment will be described with reference to the flowcharts of FIGS. FIG. 5 is a flowchart showing the operation of the broadcast apparatus 1 according to this embodiment. In this flowchart, the broadcasting unit 12 transmits the delay time information included in the sign language video information. In addition, delay time information is acquired when sign language video information is transmitted.

  (Step S101) The broadcast unit 12 determines whether to broadcast broadcast program information. And when broadcasting, it progresses to step S102, and when that is not right, it progresses to step S104.

  (Step S <b> 102) The broadcast unit 12 reads the broadcast program information frame to be transmitted from the broadcast program information storage unit 11. This frame may be, for example, one frame or a plurality of consecutive frames.

  (Step S103) The broadcast unit 12 broadcasts the read frame or frames. Then, the process returns to step S101.

  (Step S104) The sign language image information receiving unit 13 determines whether sign language image information has been received. If it has been received, the process proceeds to step S105, and if not, the process returns to step S101. The sign language video information receiving unit 13 may receive the sign language video information for each frame, or may receive it for each of a plurality of consecutive frames.

  (Step S <b> 105) The broadcasting unit 12 determines whether or not the delay time information is stored in the delay time information storage unit 15. If stored, the process proceeds to step S108. If not stored, the delay time information acquisition unit 14 is instructed to acquire delay time information, and the process proceeds to step S106.

  (Step S106) The delay time information acquisition unit 14 acquires delay time information. The delay time information acquisition unit 14 may acquire the delay time information by measuring the communication time from the sign language interpretation device 2 to the broadcast device 1 using, for example, ping.

  (Step S107) The delay time information acquisition unit 14 stores the acquired delay time information in the delay time information storage unit 15.

  (Step S108) The broadcast unit 12 reads the delay time information stored in the delay time information storage unit 15.

  (Step S109) The broadcast unit 12 includes the read delay time information in the sign language video information received by the sign language video information reception unit 13.

  (Step S110) The broadcast unit 12 broadcasts sign language video information including delay time information. Then, the process returns to step S101.

  In the flowchart of FIG. 5, the processing from steps S101 to S103 and the processing from steps S104 to S110 may be executed in parallel. In that case, broadcast of broadcast program information and broadcast of sign language video information can be executed in parallel. For example, these processes may be processed as separate threads or separate processes.

  In the flowchart of FIG. 5, the case where the delay time information is acquired when the delay time information is not stored in the delay time information storage unit 15 has been described. Before the processing from step S104 is executed, You may make it acquire delay time information. In addition, in the case of using “the start time of voice and the start time of sign language” described above, the delay time information may not be obtained even when the first sign language video information is received. In this case, information indicating that the delay time information is unknown may be included in the sign language video information and transmitted, or default delay time information may be included in the sign language video information and transmitted.

In the flowchart of FIG. 5, once the delay time information is acquired, the delay time information is not acquired thereafter. However, the delay time information is periodically acquired and the latest delay time is acquired. Information may be stored in the delay time information storage unit 15. Then, the broadcast unit 12 may transmit sign language video information including the latest delay time information.
Further, in the flowchart of FIG. 5, the process is ended by powering off or interruption for aborting the process.

FIG. 6 is a flowchart showing the operation of the sign language interpretation apparatus 2 according to this embodiment.
(Step S201) The broadcast program information receiving unit 21 determines whether broadcast program information has been received. If received, the process proceeds to step S202, and if not, the process proceeds to step S203. The broadcast program information receiving unit 21 may receive the broadcast program information for each frame, or may receive it for a plurality of consecutive frames.

  (Step S202) The video / audio output unit 22 outputs the video / audio of the broadcast program information received by the broadcast program information receiving unit 21. Then, the process returns to step S201. It is assumed that sign language interpretation is performed by a sign language interpreter who has viewed the video and audio of the output broadcast program information.

  (Step S203) The sign language video information accepting unit 23 determines whether or not a sign language interpreting video by the sign language interpreter, that is, a sign language interpreter video has been accepted. And when it receives, it progresses to step S204, and when that is not right, it returns to step S201. The sign language video information accepting unit 23 may accept the sign language video information for each frame, or may accept the sign language video information for each of a plurality of consecutive frames.

  (Step S204) The sign language video information transmitting unit 24 transmits the sign language video information received by the sign language video information receiving unit 23 to the broadcasting device 1. Then, the process returns to step S201. The sign language video information transmission unit 24 may hold the address of the broadcasting device 1 in a recording medium (not shown) in advance, and receives the address of the broadcasting device 1 from other components when transmitting sign language video information. Alternatively, the address of the broadcasting device 1 may be included in the broadcast program information, and the address may be acquired.

  In the flowchart of FIG. 6, the processing from step S201 to S202 and the processing from step S203 to S204 may be executed in parallel. In that case, reception of broadcast program information and transmission of sign language video information can be performed in parallel. For example, these processes may be processed as separate threads or separate processes.

Further, as described above, when acquiring delay time information using a time code, acquisition of a time code included in broadcast program information, transmission of sign language video information including the time code, and the like are performed. Also good.
In the flowchart of FIG. 6, the process is ended by powering off or interruption for aborting the process.

FIG. 7 is a flowchart showing the operation of the broadcast receiving apparatus 3 according to this embodiment.
(Step S301) The broadcast receiving unit 31 determines whether broadcast program information has been received. If received, the process proceeds to step S301. Otherwise, the process proceeds to step S303. Note that the broadcast receiving unit 31 may receive the broadcast program information for each frame, or may receive it for each of a plurality of consecutive frames.

  (Step S <b> 302) The broadcast program information accumulation unit 32 accumulates the broadcast program information received by the broadcast reception unit 31 in the broadcast program information storage unit 33. Then, the process returns to step S301.

  (Step S303) The broadcast receiving unit 31 determines whether or not sign language video information has been received. If sign language video information is received, the process proceeds to step S304. If not, the process returns to step S301. The broadcast receiving unit 31 may receive the sign language video information for each frame, or may receive it for a plurality of consecutive frames. Also, the broadcast receiving unit 31 can determine broadcast program information and sign language video information based on, for example, a difference in broadcast channel.

  (Step S304) The output unit 34 acquires delay time information included in the sign language video information received by the broadcast receiving unit 31.

  (Step S <b> 305) The output unit 34 reads out the broadcast program information received only by the acquired delay time information from the broadcast program information storage unit 33. When the broadcast program information and the sign language video information have the same frame rate, the output unit 34 has the same number of frames as the number of frames of the sign language video information received by the broadcast receiving unit 31 in step S303. Shall be read.

  (Step S306) The output unit 34 outputs the sign language video information received in step S303 and the broadcast program information read in step S305. Then, the process returns to step S301.

  In the flowchart of FIG. 7, the processing from step S301 to S302 and the processing from step S303 to S306 may be executed in parallel. In that case, reception of broadcast program information and output of sign language video information and broadcast program information can be executed in parallel. For example, these processes may be processed as separate threads or separate processes.

  In addition, in steps S304 and S305 in the flowchart of FIG. 7, unless the delay time information included in the sign language video information is changed, after the appropriate broadcast program information is read using the delay time information for the first time, It is only necessary to read the oldest broadcast program information stored in the broadcast program information storage unit 33 and not yet output. Therefore, in the sign language interpretation system in which the delay time information is not frequently changed, the process of step S304 and the process of acquiring the broadcast program information in step S305 using the delay time information acquired in step S304 are performed. The process is executed only once, and thereafter, the process of reading the oldest broadcast program information stored in the broadcast program information storage unit 33 and not yet output is performed in step S305. Also good. Also in that case, the processing of step S304 for acquiring delay time information and the processing of step S305 for acquiring broadcast program information using the acquired delay time information may be performed for each predetermined period. .

Also, in the sign language interpretation system in which the delay time information changes, if the delay time information suddenly shortens, there may be a frame of broadcast program information that is not output. Thus, even if the delay time information suddenly becomes short, the broadcast program information and the delay time information are synchronized by fast-forwarding only the broadcast program information so that a frame of broadcast program information that is not output does not occur. In addition, the frame of the broadcast program information may not be dropped. On the contrary, when the delay time information becomes long, there is a possibility that there is no frame of broadcast program information to be output. In such a case, time may be earned by slowly outputting frames of broadcast program information or by continuously outputting the same frames.
In the flowchart of FIG. 7, the process ends when the power is turned off or the process is terminated.

  Next, the operation of the sign language interpretation system according to this embodiment will be described using a specific example. In this specific example, a case where delay time information is acquired using a time code will be described. The sign language delay time is set to 3 seconds in advance.

  First, when the broadcast unit 12 broadcasts the broadcast program information of the weather forecast, the first frame of the broadcast program information is read (steps S101 and S102), and the time code “00 (hour): 00 ( Assume that the broadcast is performed with "minutes: 00 (seconds): 01 (frames)" (step S103).

  Here, when the first frame of the broadcast program information of the weather forecast is broadcast, the delay time information acquisition unit 14 sets the time “10 (hours): 00 (minutes): 00.00 ( Second) ”and the time code“ 00: 00: 00: 01 ”corresponding to the frame are temporarily stored in a recording medium (not shown).

  After this, for each frame, the reading of the frame, the addition of the time code to the frame, and the broadcast of the frame are sequentially performed (steps S101 to S103). Also, the time at which the frame is broadcast and the time code of the frame are sequentially associated and stored as shown in FIG.

  Next, it is assumed that the sign language interpreter 2 has previously set a channel so that the sign language interpreter can receive the broadcast program information of the weather forecast transmitted from the broadcast device 1. Then, the broadcast program information receiving unit 21 receives the weather forecast broadcast program information frame transmitted from the broadcast device 1, passes the time code included in the frame to the sign language video information transmitting unit 24, and transmits the frame. The video / audio output unit 22 is handed over (step S201). Then, the video / audio output unit 22 outputs the video and audio of the weather forecast broadcast program information (step S202). For example, the video shown in FIG. 9 is displayed on the display. Note that the reception of the broadcast program information and the output of video and audio are also repeatedly performed in sequence (steps S201 and S202).

  In addition, it is assumed that the sign language interpreter has made settings such that sign language video information, which is a video taken by the camera, is transmitted to the broadcasting device 1. Then, sign language video information from the camera is received by the sign language video information receiving unit 23 (step S203). After that, the sign language video information transmission unit 24 adds the latest time code “00: 00: 01: 00” received from the broadcast program information reception unit 21 to the frame of the sign language video information received by the sign language video information reception unit 23. Assigned and transmitted to the broadcasting device 1 (step S204). Note that the reception of the sign language video information and the transmission of the sign language video information are also sequentially repeated (steps S203 and S204). Also, the sign language interpreter views the weather forecast video and audio output by the video / audio output unit 22 and performs sign language interpretation corresponding to the audio, so that the sign language video is sequentially transmitted to the broadcasting device 1. Will be. For example, sign language video information that is an image of a sign language interpreter shown in FIG. 10 is transmitted to the broadcasting device 1.

  The sign language video information frame transmitted from the sign language interpreting apparatus 2 is received by the sign language video information receiving unit 13, and the frame is delivered to the broadcasting unit 12 and the time code “00:00: 00:01 "is passed to the delay time information acquisition unit 14 (step S104). At this point, since the delay time information is not stored in the delay time information storage unit 15, the broadcast unit 12 passes an instruction to the delay time information acquisition unit 14 to acquire the delay time information (step S105). Then, the delay time information acquisition unit 14 refers to the information associating the time and the time code shown in FIG. 8 and corresponds to the time code “00: 00: 01: 00” received from the sign language video information reception unit 13. The time “10: 00: 00.00” and the current time “10: 00: 01: 00” are acquired. Then, the difference “1.00 seconds” between the two times is calculated. This time is the communication delay time. Then, the delay time information “4 seconds” is calculated by adding the interpretation delay time “3 seconds” held in advance and the communication delay time “1 second” (step S106). Then, the delay time information acquisition unit 14 accumulates the delay time information “4 seconds” in the delay time information storage unit 15 (step S107).

  Thereafter, the broadcast unit 12 reads the delay time information “4 seconds” from the delay time information storage unit 15 (step S108), and the delay time information “4 seconds” is added to the frame of the sign language video information received by the sign language video information reception unit 13. "Is added and broadcast (steps S109 and S110). It is assumed that the broadcast unit 12 broadcasts a frame of sign language video information with a time code, as in the case of broadcasting a frame of broadcast program information. The time code is the same as the time code of the broadcast program information. Therefore, the same time code is given to the broadcast program information frame and the sign language video information frame broadcast simultaneously from the broadcast apparatus 1. Also, the reception of the sign language video information and the broadcast of the sign language video information to which the delay time information is added are sequentially repeated (steps S104, S105, S108 to S110).

  It is assumed that the user of the broadcast receiving device 3 has set a channel so that the weather forecast broadcast program information transmitted from the broadcast device 1 can be received. Then, each frame of the broadcast program information broadcast from the broadcast device 1 is received by the broadcast receiving unit 31 (step S301) and sequentially stored in the broadcast program information storage unit 33 (step S302).

  Also, sign language video information broadcast from the broadcasting device 1 is received by the broadcast receiving unit 31 (step S303). The output unit 34 acquires the delay time information “4 seconds” from the frame of the sign language video information (step S304). Then, by reading out from the broadcast program information storage unit 33 the broadcast program information frame to which the time code 4 seconds before the time code given to the received sign language video information frame is added, the output unit 34 can read the frame of the broadcast program information four seconds ago (step S305). Thereafter, the output unit 34 outputs the broadcast program information frame and the sign language video information frame (step S306).

  FIG. 11 is a diagram for explaining synchronization between sign language video information and broadcast program information. In FIG. 11, the upper side shows each frame of received broadcast program information, and the lower side shows each frame of received sign language video information. The time code assigned to each frame is displayed below each frame. For example, when the frame BFN of sign language video information is received by the broadcast receiving unit 31, the output unit 34 outputs the time code “00” that is 4 seconds before the time code “00: 00: 10: 01” of the frame BFN. : 0: 06: 01 "is read from the broadcast program information storage unit 33, and both are output.

  Note that the frame output by the output unit 34 may be one that displays a sign language video information frame within a broadcast program information frame, as shown in FIG. 12, or as shown in FIG. As shown, the frame of broadcast program information and the frame of sign language video information may be displayed side by side.

  In this specific example, the case where the sign language delay time is constant has been described, but the sign language delay time may be determined for each sign language interpreter. For example, it is assumed that the delay time information acquisition unit 14 can access information associating sign language interpreter identification information with sign language delay time shown in FIG. Further, it is assumed that the sign language video information transmitted from the sign language interpreting apparatus 2 includes sign language interpreter identification information that is information for identifying a sign language interpreter who has performed sign language interpretation related to the sign language video information. Then, the delay time information acquisition unit 14 acquires the sign language delay time corresponding to the sign language interpreter identification information included in the sign language video information using the information shown in FIG. 14, and uses the sign language delay time. The delay time information may be calculated. Note that the information shown in FIG. 14 may be stored in a recording medium (not shown) included in the delay time information acquisition unit 14, or may be stored in a recording medium included in other components. In addition, the sign language interpreter identification information transmitted from the sign language interpreting apparatus 2 may be included in the sign language video information, or may be transmitted separately from the sign language video information. When the sign language video information transmitted from the sign language interpreter 2 includes sign language interpreter identification information, for example, the sign language interpreter 2 stores a sign language interpreter identification information storage unit in which the sign language interpreter identification information is stored. The sign language video information transmitting unit 24 may read the sign language interpreter identification information stored in the sign language interpreter identification information storage unit and include it in the sign language video information for transmission.

  Further, in this specific example, when the broadcasting device 1 cannot receive the sign language video information due to the malfunction of the communication line 500 or the sign language interpreting device 2, information indicating the cause thereof is used as the sign language video information. May be broadcast. For example, “Signature video information cannot be broadcast due to equipment malfunction. Sorry for the inconvenience, but thank you for your understanding.” You may do it. By doing so, the user of the broadcast receiving apparatus 3 can know that the reason why the sign language video information cannot be seen exists on the broadcasting apparatus 1 side and is not caused by the malfunction of the broadcast receiving apparatus 3. .

  As described above, according to the sign language interpreting system according to the present embodiment, broadcast program information can be broadcast, and sign language interpretation can be performed in the sign language interpreting apparatus 2, that is, on the network. It is also possible to broadcast sign language video information, which is a recorded video. Accordingly, a hearing impaired person can view both broadcast program information and sign language video information only by preparing a device for receiving a broadcast program such as a television receiver. As a result, there is an advantage that both pieces of information can be easily grasped at a glance as compared with a case where a device receiving broadcast program information and a device receiving sign language video information are separate.

  Further, by synchronizing and outputting the broadcast program information and the sign language video information using the delay time information, the broadcast program information and the sign language video information output from the broadcast receiving device 3 correspond in time, A hearing-impaired person can understand the broadcast program more accurately, and the convenience of the hearing-impaired person can be improved.

  Furthermore, by acquiring the delay time information by the delay time information acquisition unit 14 of the broadcasting device 1, it is possible to use the delay time information according to the situation at that time, and more accurately the broadcast program information and the sign language video information. It can be synchronized. For example, when the degree of congestion of the communication line 500 changes depending on the time zone, it is possible to change the daytime delay time information and the nighttime delay time information by acquiring the delay time information for each broadcast program information. And more precise synchronization is possible.

  Furthermore, by broadcasting broadcast program information and sign language video information on separate channels, only those who need sign language video information can output the sign language video information along with the broadcast program information, and the sign language video information is required. Those who do not can output only broadcast program information. Therefore, it is possible to prevent a person who does not need sign language video information from showing sign language video information, and a necessary and sufficient video can be output according to the user.

  In the present embodiment, the case where the delay time information acquisition unit 14 acquires the delay time information and stores it in the delay time information storage unit 15 in the broadcasting device 1 has been described, but this need not be the case. The delay time information may be stored in advance in the delay time information storage unit 15. In that case, the broadcasting device 1 may not include the delay time information acquisition unit 14. In this case, the process for storing the delay time information in the delay time information storage unit 15 is not limited. For example, the delay time information may be stored in the delay time information storage unit 15 via a recording medium, and the delay time information transmitted via a communication line or the like is stored in the delay time information storage unit 15. Alternatively, the delay time information input via the input device may be stored in the delay time information storage unit 15.

  In the present embodiment, the case where the broadcast unit 12 of the broadcast apparatus 1 also broadcasts the delay time information stored in the delay time information storage unit 15 has been described, but this need not be the case. The broadcast unit 12 may not broadcast the delay time information. In that case, the broadcasting device 1 may not include the delay time information storage unit 15 and the delay time information acquisition unit 14. Further, the output unit 34 of the broadcast receiving device 3 may not perform the process of synchronizing the broadcast program information and the sign language video information using the delay time information. Even if the broadcast program information and sign language video information are output without being synchronized, it is considered that there is a time lag of several seconds, so it is considered that the content of the broadcast program information can be understood to some extent by the sign language video information. .

  In the above embodiment, each process or each function may be realized by centralized processing by a single device or a single system, or may be distributedly processed by a plurality of devices or a plurality of systems. It may be realized by doing.

  In the above embodiment, information related to processing executed by each component, for example, information received, acquired, selected, generated, transmitted, or received by each component In addition, information such as threshold values, mathematical formulas, addresses, etc. used by each component in processing is retained temporarily or over a long period of time on a recording medium (not shown) even when not explicitly stated in the above description. It may be. Further, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Further, reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  In the above embodiment, when information used by each component, for example, information such as a threshold value, an address, and various setting values used by each component may be changed by the user Even if it is not specified in the above description, the user may be able to change the information as appropriate, or it may not be. If the information can be changed by the user, the change is realized by, for example, a not-shown receiving unit that receives a change instruction from the user and a changing unit (not shown) that changes the information in accordance with the change instruction. May be. The change instruction received by the receiving unit (not shown) may be received from an input device, information received via a communication line, or information read from a predetermined recording medium, for example. .

  In the above embodiment, when two or more components included in the sign language interpretation system include a communication device, an input device, or the like, the two or more components may physically include a single device. Alternatively, it may have a separate device.

  In the above embodiment, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. In addition, the software which implement | achieves the broadcasting apparatus 1 in the said embodiment is the following programs. In other words, this program is a program for causing a computer to function as a broadcasting device constituting a sign language interpretation system including a broadcasting device and a sign language interpretation device, and is an image of sign language transmitted from the sign language interpretation device. Broadcasts broadcast program information stored in a sign language video information receiving unit that receives sign language video information that is information indicating a broadcast program information storage unit that stores broadcast program information that is information indicating a program to be broadcast The sign language video information received by the sign language video information receiving unit functions as a broadcasting unit that also broadcasts the sign language video information received by the sign language video information receiving unit, and the sign language video information received by the sign language video information receiving unit corresponds to the broadcast program information broadcast by the broadcasting unit. It is a program.

  Moreover, the software which implement | achieves the sign language interpretation apparatus 2 in the said embodiment is the following programs. That is, this program is a program for causing a computer to function as a broadcasting device that constitutes a sign language interpretation system including a broadcasting device and a sign language interpretation device, and receives broadcast program information broadcast by the broadcasting device. Broadcast program information receiving unit, video / audio output unit for outputting video / audio of broadcast program information received by the broadcast program information receiving unit, and information indicating sign language video corresponding to the video / audio output by the video / audio output unit A sign language video information receiving unit that receives the sign language video information, and a sign language video information transmitting unit that transmits the sign language video information received by the sign language video information receiving unit to the broadcasting device.

  Moreover, the software which implement | achieves the broadcast receiver 3 in the said embodiment is the following programs. That is, the program receives the broadcast program information broadcast by the broadcast device, and the sign language video information corresponding to the broadcast program information broadcast by the broadcast device and the broadcast program information broadcast by the broadcast device. A broadcast receiving unit that receives delay time information that is information indicating a time during which sign language video information is delayed, a video of sign language video information received by the broadcast receiving unit, and a delay received by the broadcast receiving unit It is a program for functioning as an output unit for outputting video of broadcast program information received by the broadcast receiving unit delayed by a delay time indicated by time information.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, functions that can be realized only by hardware such as a modem and an interface card in an acquisition unit that acquires information, a transmission unit that transmits information, and the like are not included at least in the functions realized by the program.

  Further, this program may be executed by being downloaded from a server or the like, and a program recorded on a predetermined recording medium (for example, an optical disk such as a CD-ROM, a magnetic disk, a semiconductor memory, or the like) is read out. May be executed by Further, this program may be used as a program constituting a program product.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 15 is a schematic diagram showing an example of the external appearance of a computer that executes the above program and realizes the broadcasting device 1, the sign language interpreting device 2, and the broadcasting receiving device 3 according to the above embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  15, a computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905, an FD (Floppy (registered trademark) Disk) drive 906, a keyboard 902, a mouse 903, a monitor 904, and the like. Is provided.

  FIG. 16 is a diagram showing an internal configuration of the computer system 900. In FIG. 16, in addition to the CD-ROM drive 905 and the FD drive 906, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores program instructions and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and an MPU 911 and a ROM 912 are interconnected. And a bus 915. Note that the computer 901 may include a network card (not shown) that provides connection to the LAN, a device that broadcasts information, a device that receives broadcasts, and the like.

  A program that causes the computer system 900 to execute the functions of the broadcasting device 1, the sign language interpretation device 2, and the broadcast receiving device 3 according to the above-described embodiment is stored in the CD-ROM 921 or FD 922, and the CD-ROM drive 905 or FD. It may be inserted into the drive 906 and transferred to the hard disk 914. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921, the FD 922, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 901 to execute the functions of the broadcasting device 1, the sign language interpreting device 2, and the broadcasting receiving device 3 according to the above-described embodiment. . The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  Further, the present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, according to the sign language interpretation system or the like according to the present invention, a broadcast program and a sign language video corresponding to the broadcast program can be received by one broadcast receiving device, and a broadcast program and a sign language video can be transmitted and received. This is useful as a sign language interpretation system.

1 is a block diagram showing the configuration of a sign language interpretation system according to Embodiment 1 of the present invention. The block diagram which shows the structure of the broadcasting apparatus by the embodiment The block diagram which shows the structure of the sign language interpretation apparatus by the embodiment The block diagram which shows the structure of the broadcast receiving apparatus by the embodiment The flowchart which shows operation | movement of the broadcasting apparatus by the embodiment The flowchart which shows operation | movement of the sign language interpreting apparatus by the embodiment The flowchart which shows operation | movement of the broadcast receiving apparatus by the embodiment The figure which shows an example of the information used in order to acquire the delay time information in the embodiment The figure which shows an example of the screen of the broadcast program information in the embodiment The figure which shows an example of the screen of the sign language image information in the embodiment The figure for demonstrating the synchronization with broadcast program information and sign language video information in the embodiment The figure which shows an example of the screen of broadcast program information and sign language video information in the embodiment The figure which shows an example of the screen of broadcast program information and sign language video information in the embodiment The figure which shows an example of the information used in order to acquire the delay time information in the embodiment Schematic diagram showing an example of the appearance of the computer system in the embodiment The figure which shows an example of a structure of the computer system in the embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Broadcasting apparatus 2 Sign language interpretation apparatus 3 Broadcast receiving apparatus 11, 33 Broadcast program information storage part 12 Broadcasting part 13 Sign language video information receiving part 14 Delay time information acquisition part 15 Delay time information storage part 21 Broadcast program information receiving part 22 Video / audio output 23 Sign language video information receiving unit 24 Sign language video information transmitting unit 31 Broadcast receiving unit 32 Broadcast program information accumulating unit 34 Output unit

Claims (15)

A sign language interpretation system comprising a broadcasting device and a sign language interpretation device,
The broadcasting device is:
A sign language video information receiving unit that receives sign language video information that is transmitted from the sign language interpretation device and that indicates video of a sign language;
A broadcast program information storage unit for storing broadcast program information which is information indicating a program to be broadcast;
A broadcast unit that broadcasts the broadcast program information stored in the broadcast program information storage unit and also broadcasts the sign language video information received by the sign language video information reception unit;
A delay time information storage unit for storing delay time information that is information indicating a time during which sign language video information is delayed with respect to broadcast program information broadcast by the broadcast unit;
Measure the communication time of information between the broadcasting device and the sign language interpreting device, and acquire delay time information indicating the time obtained by adding the communication time and a pre-held sign language interpretation time difference A delay time information acquisition unit that accumulates in the delay time information storage unit ,
The broadcast unit also broadcasts the delay time information stored in the delay time information storage unit,
The sign language interpreter is
A broadcast program information receiving unit for receiving broadcast program information broadcast by the broadcast device;
A video / audio output unit for outputting video / audio of broadcast program information received by the broadcast program information receiving unit;
A sign language video information receiving unit that receives sign language video information that is information indicating a video of sign language corresponding to the video and audio output by the video and audio output unit;
A sign language interpretation system, comprising: a sign language video information transmission unit that transmits sign language video information received by the sign language video information reception unit to the broadcasting device. A sign language interpretation system comprising a broadcasting device and a sign language interpretation device,
The broadcasting device is:
A sign language video information receiving unit that receives sign language video information that is transmitted from the sign language interpretation device and that indicates video of a sign language;
A broadcast program information storage unit for storing broadcast program information which is information indicating a program to be broadcast;
A broadcast unit that broadcasts the broadcast program information stored in the broadcast program information storage unit and also broadcasts the sign language video information received by the sign language video information reception unit;
A delay time information storage unit for storing delay time information that is information indicating a time during which sign language video information is delayed with respect to broadcast program information broadcast by the broadcast unit;
A delay time information acquisition unit that acquires the delay time information and accumulates it in the delay time information storage unit,
The sign language interpreter is
A broadcast program information receiving unit for receiving broadcast program information broadcast by the broadcast device;
A video / audio output unit for outputting video / audio of broadcast program information received by the broadcast program information receiving unit;
A sign language video information receiving unit that receives sign language video information that is information indicating a video of sign language corresponding to the video and audio output by the video and audio output unit;
A sign language video information transmission unit that transmits the sign language video information received by the sign language video information reception unit to the broadcasting device,
The broadcast unit broadcasts broadcast program information including a time code, and also broadcasts delay time information stored in the delay time information storage unit,
The sign language video information transmitting unit transmits sign language video information including the latest time code included in the broadcast program information received by the broadcast program information receiving unit;
The delay time information acquisition unit is a time period from when the broadcast unit broadcasts broadcast program information including a certain time code to when the sign language video information reception unit receives sign language video information including the time code. get the communication time, to obtain the corresponding communication time, the delay time information indicating the time adding the advance the held shift time for sign language interpreters, sign language interpretation system. A sign language interpretation system comprising a broadcasting device and a sign language interpretation device,
The broadcasting device is:
A sign language video information receiving unit that receives sign language video information that is transmitted from the sign language interpretation device and that indicates video of a sign language;
A broadcast program information storage unit for storing broadcast program information which is information indicating a program to be broadcast;
A broadcast unit that broadcasts the broadcast program information stored in the broadcast program information storage unit and also broadcasts the sign language video information received by the sign language video information reception unit;
A delay time information storage unit for storing delay time information that is information indicating a time during which sign language video information is delayed with respect to broadcast program information broadcast by the broadcast unit;
From the start of the sound the broadcast unit is included in the broadcast program information broadcast, and obtains the delay time information indicating the time until the beginning of the sign language video indicated by the sign language video information said sign language video information receiving unit receives A delay time information acquisition unit that accumulates in the delay time information storage unit,
The broadcast unit also broadcasts the delay time information stored in the delay time information storage unit,
The sign language interpreter is
A broadcast program information receiving unit for receiving broadcast program information broadcast by the broadcast device;
A video / audio output unit for outputting video / audio of broadcast program information received by the broadcast program information receiving unit;
A sign language video information receiving unit that receives sign language video information that is information indicating a video of sign language corresponding to the video and audio output by the video and audio output unit;
A sign language interpretation system , comprising: a sign language video information transmission unit that transmits sign language video information received by the sign language video information reception unit to the broadcasting device . The sign language interpreting system according to any one of claims 1 to 3, wherein the broadcasting section broadcasts the sign language video information including the delay time information. A broadcast receiving device;
The broadcast receiving device is:
A broadcast receiver for receiving broadcast program information, sign language video information and delay time information broadcast by the broadcast device;
An output unit that outputs the video of the sign language video information received by the broadcast receiver and the video of the broadcast program information delayed by a delay time indicated by the delay time information received by the broadcast receiver; The sign language interpreting system according to any one of claims 1 to 4 . A sign language interpreting device constituting the sign language interpreting system according to any one of claims 1 to 5 . A broadcasting device that constitutes a sign language interpretation system including a broadcasting device and a sign language interpretation device,
A sign language video information receiving unit that receives sign language video information that is transmitted from the sign language interpretation device and that indicates video of a sign language;
A broadcast program information storage unit storing broadcast program information which is information indicating a program to be broadcast;
A broadcast unit that broadcasts the broadcast program information stored in the broadcast program information storage unit and also broadcasts the sign language video information received by the sign language video information reception unit;
A delay time information storage unit for storing delay time information that is information indicating a time during which sign language video information is delayed with respect to broadcast program information broadcast by the broadcast unit;
Measure the communication time of information between the broadcasting device and the sign language interpreting device, and acquire delay time information indicating the time obtained by adding the communication time and a pre-held sign language interpretation time difference A delay time information acquisition unit that accumulates in the delay time information storage unit ,
The broadcast unit also broadcasts the delay time information stored in the delay time information storage unit,
The sign device image information received by the sign language image information receiving unit corresponds to broadcast program information broadcast by the broadcasting unit. A broadcasting device that constitutes a sign language interpretation system including a broadcasting device and a sign language interpretation device,
A sign language video information receiving unit that receives sign language video information that is transmitted from the sign language interpretation device and that indicates video of a sign language;
A broadcast program information storage unit storing broadcast program information which is information indicating a program to be broadcast;
A broadcast unit that broadcasts the broadcast program information stored in the broadcast program information storage unit and also broadcasts the sign language video information received by the sign language video information reception unit;
A delay time information storage unit for storing delay time information that is information indicating a time during which sign language video information is delayed with respect to broadcast program information broadcast by the broadcast unit;
A delay time information acquisition unit that acquires the delay time information and accumulates it in the delay time information storage unit,
The broadcast unit broadcasts broadcast program information including a time code, and also broadcasts delay time information stored in the delay time information storage unit,
The sign language video information received by the sign language video information receiving unit corresponds to broadcast program information broadcast by the broadcasting unit,
The sign language video information receiving unit receives sign language video information including the latest time code included in the broadcast program information received by the sign language interpretation device;
The delay time information acquisition unit is a time period from when the broadcast unit broadcasts broadcast program information including a certain time code to when the sign language video information reception unit receives sign language video information including the time code. A broadcast apparatus that acquires communication time and acquires delay time information indicating a time obtained by adding the communication time and a preliminarily held shift time during sign language interpretation. A broadcasting device that constitutes a sign language interpretation system including a broadcasting device and a sign language interpretation device,
A sign language video information receiving unit that receives sign language video information that is transmitted from the sign language interpretation device and that indicates video of a sign language;
A broadcast program information storage unit storing broadcast program information which is information indicating a program to be broadcast;
A broadcast unit that broadcasts the broadcast program information stored in the broadcast program information storage unit and also broadcasts the sign language video information received by the sign language video information reception unit;
A delay time information storage unit for storing delay time information that is information indicating a time during which sign language video information is delayed with respect to broadcast program information broadcast by the broadcast unit;
Obtaining delay time information indicating a time from a start time of sound included in broadcast program information broadcasted by the broadcast unit to a start time of sign language of the video indicated by the sign language video information received by the sign language video information receiving unit; A delay time information acquisition unit that accumulates in the delay time information storage unit,
The broadcast unit also broadcasts the delay time information stored in the delay time information storage unit,
The sign device image information received by the sign language image information receiving unit corresponds to broadcast program information broadcast by the broadcasting unit. A broadcasting method used in a broadcasting device constituting a sign language interpretation system including a broadcasting device and a sign language interpretation device,
The broadcast apparatus, and the broadcast program information broadcasting step of broadcasting the broadcast program information broadcast program information storage unit the broadcast program information is information indicating a program to be broadcast is stored is stored,
The broadcast apparatus, the sent from sign language interpreter device, a sign language video information receiving step of receiving the sign language video information indicating the sign language video corresponding to the broadcast program information broadcast on the broadcast program information broadcasting step,
A sign language video information broadcasting step in which the broadcasting device broadcasts the sign language video information received in the sign language video information receiving step;
The delay in which the broadcast device acquires delay time information that is information indicating the time that the sign language video information is delayed with respect to the broadcast program information broadcast in the broadcast program information broadcast step and accumulates it in the delay time information storage unit A time information acquisition step ,
In the delay time information acquisition step, a communication time of information between the broadcasting device and the sign language interpreting device is measured, and a time obtained by adding the communication time and a pre-stored sign language interpreting time is calculated. Get the delay time information shown,
In the sign language video information broadcasting step, the delay time information stored in the delay time information storage unit is also broadcast . A broadcasting method used in a broadcasting device constituting a sign language interpretation system including a broadcasting device and a sign language interpretation device,
A broadcast program information broadcast step for broadcasting the broadcast program information stored in a broadcast program information storage unit in which broadcast program information that is information indicating a program to be broadcast is stored;
A sign language video information receiving step for receiving sign language video information, which is information indicating a sign language video corresponding to the broadcast program information broadcasted in the broadcast program information broadcast step, transmitted from the sign language interpreting device;
A sign language video information broadcasting step in which the broadcasting device broadcasts the sign language video information received in the sign language video information receiving step;
The delay in which the broadcast device acquires delay time information that is information indicating the time that the sign language video information is delayed with respect to the broadcast program information broadcast in the broadcast program information broadcast step and accumulates it in the delay time information storage unit A time information acquisition step,
In the broadcast program information broadcast step, broadcast program information including a time code is broadcast,
In the sign language video information broadcasting step, the delay time information stored in the delay time information storage unit is also broadcast,
In the sign language video information receiving step, the sign language video information including the latest time code included in the broadcast program information received by the sign language interpretation device is received,
In the delay time information acquisition step, from the time when broadcast program information including a certain time code is broadcast in the broadcast program information broadcasting step to the time when sign language video information including the time code is received in the sign language video information receiving step. A broadcast method of acquiring a communication time as a time, and acquiring delay time information indicating a time obtained by adding the communication time and a preliminarily held shift time during sign language interpretation. A broadcasting method used in a broadcasting device constituting a sign language interpretation system including a broadcasting device and a sign language interpretation device,
A broadcast program information broadcast step for broadcasting the broadcast program information stored in a broadcast program information storage unit in which broadcast program information that is information indicating a program to be broadcast is stored;
A sign language video information receiving step for receiving sign language video information, which is information indicating a sign language video corresponding to the broadcast program information broadcasted in the broadcast program information broadcast step, transmitted from the sign language interpreting device;
A sign language video information broadcasting step in which the broadcasting device broadcasts the sign language video information received in the sign language video information receiving step;
The delay in which the broadcast device acquires delay time information that is information indicating the time that the sign language video information is delayed with respect to the broadcast program information broadcast in the broadcast program information broadcast step and accumulates it in the delay time information storage unit A time information acquisition step,
In the sign language video information broadcasting step, the delay time information stored in the delay time information storage unit is also broadcast,
In the delay time information acquisition step, from the start time of the audio included in the broadcast program information broadcast in the broadcast program information broadcast step to the start time of the sign language of the video indicated by the sign language video information received in the sign language video information reception step A broadcast method for acquiring delay time information indicating the time of the broadcast. Computer
A program for functioning as a broadcasting device constituting a sign language interpretation system comprising a broadcasting device and a sign language interpretation device,
A sign language video information receiving unit that receives sign language video information that is transmitted from the sign language interpretation device and that indicates video of a sign language;
Broadcast that broadcasts broadcast program information stored in a broadcast program information storage unit that stores broadcast program information that is information indicating a program to be broadcast, and also broadcasts sign language video information received by the sign language video information receiving unit and parts,
As the delay time information acquisition unit that the broadcast unit is accumulated in the delay time information storage unit acquires the delay time information sign language video information indicates the time that is delayed with respect to the broadcast program information broadcasting Make it work
Sign language video information said sign language video information receiving unit receives the state, and are not corresponding to the broadcast program information the broadcast unit is broadcasting,
The broadcast unit also broadcasts the delay time information stored in the delay time information storage unit,
The delay time information acquisition unit measures a communication time of information between the broadcasting device and the sign language interpreting device, and adds a time obtained by adding the communication time and a preliminarily held shift time during sign language interpretation. you get the delay time information indicating, program. Computer
A program for functioning as a broadcasting device constituting a sign language interpretation system comprising a broadcasting device and a sign language interpretation device,
A sign language video information receiving unit that receives sign language video information that is transmitted from the sign language interpretation device and that indicates video of a sign language;
Broadcast that broadcasts broadcast program information stored in a broadcast program information storage unit that stores broadcast program information that is information indicating a program to be broadcast, and also broadcasts sign language video information received by the sign language video information receiving unit And
It functions as a delay time information acquisition unit that acquires delay time information, which is information indicating a time during which sign language video information is delayed with respect to broadcast program information broadcast by the broadcast unit, and accumulates it in the delay time information storage unit. ,
The sign language video information received by the sign language video information receiving unit corresponds to broadcast program information broadcast by the broadcasting unit,
The broadcast unit broadcasts broadcast program information including a time code, and also broadcasts delay time information stored in the delay time information storage unit,
The sign language video information receiving unit receives sign language video information including the latest time code included in the broadcast program information received by the sign language interpretation device;
The delay time information acquisition unit is a time period from when the broadcast unit broadcasts broadcast program information including a certain time code to when the sign language video information reception unit receives sign language video information including the time code. A program for acquiring a communication time and acquiring delay time information indicating a time obtained by adding the communication time and a preliminarily held shift time during sign language interpretation. Computer
A program for functioning as a broadcasting device constituting a sign language interpretation system comprising a broadcasting device and a sign language interpretation device,
A sign language video information receiving unit that receives sign language video information that is transmitted from the sign language interpretation device and that indicates video of a sign language;
Broadcast that broadcasts broadcast program information stored in a broadcast program information storage unit that stores broadcast program information that is information indicating a program to be broadcast, and also broadcasts sign language video information received by the sign language video information receiving unit And
It functions as a delay time information acquisition unit that acquires delay time information, which is information indicating a time during which sign language video information is delayed with respect to broadcast program information broadcast by the broadcast unit, and accumulates it in the delay time information storage unit. ,
The sign language video information received by the sign language video information receiving unit corresponds to broadcast program information broadcast by the broadcasting unit,
The broadcast unit also broadcasts the delay time information stored in the delay time information storage unit,
The delay time information acquisition unit calculates a time from the start time of the sound included in the broadcast program information broadcast by the broadcast unit to the start time of the sign language of the video indicated by the sign language video information received by the sign language video information reception unit. A program that obtains delay time information to indicate.

Images