JP2773647B2 - Video display device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for displaying a signal transmitted from a partner connected through a communication path with a limited transmission amount using a video image so that a user can easily understand the signal. The present invention relates to a video display device that displays a video on a telephone, an electronic conference system, a guide terminal, a video robot, and the like.

[0002]

2. Description of the Related Art Conventionally, when a signal sent from a partner connected through a communication path is displayed on a display device such as a display, the signal is captured via an interface 10 as shown in FIG. When the signal transmitted from the transmitting side is video image data by decoding by the converting means 11 and displaying on the display means 12, it is displayed as it is on the receiving side as a video image.
In this method, a signal transmitted from the transmission side is video image data, and a video image in a monitoring system, a videophone, or the like corresponds to this. For example, in a plant control system or the like, a surveillance camera captures a situation in a plant with a camera and transmits a video image thereof, and the terminal side displays the video image on a display screen. For such systems, for example, AC
Reference 1 "CHI'9" of M Press (ACM Press)
2 Confedence Proceedings
593 to 598: Tani, Tanikoshi, Futagawa, Tanifuji Ob
JECT-ORIENTED VIDEO: Inter
action with Real-world Obj
ects Through Live Video ".

In a videophone or the like, a video image of a face on the transmitting side is transmitted through a communication channel, and the receiving side displays the video image on a display screen. If the video signal is encoded, it is decoded and displayed. Such a system and an encoding method are described in, for example, Reference 2 “Television Society Journal, VOL42 No. 6 (1988) Harashima: Pages 519 to 525, Intelligent Image Coding and Intelligent Communication”. Also in the electronic conference system using a computer, reference 3 "Transactions of Information Processing Society of Japan, V
OL32 No9 (September 1991) It is shown that such a video image is transmitted to Watanabe, Sakata, Maeno, Fukuoka, Omori: Multimedia distributed enrollment conference system MERMAID. For this, a video image is input from a video camera and 64 Kb / s by a video codec.
It is described that the data is compressed and transmitted to a communication path, and the receiving side decodes the data and displays it on a display screen.

When the signal sent from the transmitting side is a text string such as ASCII characters, the information fetched by the interface 10 is processed by the display information creating means 13 as shown in FIG. It is displayed as it is, or the color of the button displayed on the screen is changed according to the character string. As an example of this, for example, in a process computer system of a nuclear power plant, it is described in the document 4 “Measurement and control VOL30 No. 7” that the alarm status of the plant is displayed / printed on the output terminal sequentially with the alarm content and the current value of the corresponding channel.
(July 1991) Kato, Fukuchi: Nuclear Power Plant Computer System, pp. 573-581.

[0005] In order to enable the operator to intuitively grasp a large amount of data in the process monitoring work, there is a case where measured values are sometimes shown in a graph, which is described in Document 5 "Measurement and Control VOL32 No. 3 (March 1993)". Mon) Okamura:
Human Interface in an Industrial Machine System, page 205 to page 211 ". Conventionally, visualization of the transmitted data using computer graphics technology is easy to understand.

Reference 6 “Computer software
VOL10 No1 Calendar Book: CSCW Infrastructure System (Three-way talk) and its Simultaneous Effective Control Method from page 51 to page 56 "
In a real-time electronic conferencing system, a workstation at a location away from the coordinates of the mouse cursor used as an input device by a person when multiple workers perform the same task on workstations at different locations Describes a telepointer to be displayed on the screen.

[0007]

When a video image is transmitted, there is an advantage that the user can intuitively grasp the state of the other party. On the other hand, a high-speed and large-capacity transmission of the video image is possible. Communication path, or high-performance coding /
Requires a decryption device. For example, in the above-mentioned “Multimedia Distributed Enrollment Conference System MERMAID”, it is described that by capturing and sending the faces of the participants in a video image, they can see each other's faces to increase the intimacy of the participants. However, an encoding / decoding device for compressing a video image to 64 Kb / s and an ISDN line as a channel for transmitting the same are required. When the signal from the transmitting side is transmitted as a text string such as an ASCII character string, the request for the communication path may be lower than that in the case of the video image, but the receiving side receives the signal as it is. When displayed, it becomes a very confusing sequence of numbers and letters.

Further, when these signals are converted into a graphic display and displayed, not only a considerable amount of calculation is required for the graphic display but also a case where a video image is received and displayed. In addition, the display with a sense of reality cannot be performed. For example, as an example of a process monitoring system, the "nuclear power plant computer system" sequentially displays and prints the alarm contents of the plant and the current value of the corresponding channel, and the "human interface in the industrial machine system" graphs the measured values. Although it is possible to display the state of the monitoring target in a realistic video image based on the transmitted data, it is necessary to use a high-speed, large-capacity communication path or a high-performance encoding / decoding device. If there is no, it cannot be implemented. Or fast
Even when a large-capacity communication channel is used in a time-division manner, if a small number of time slots are allocated, it takes time to display. In addition, as described in “CSCW Infrastructure System (Three-way talk) and Simultaneous Effective Control Method”, a telepointer that displays the coordinates of the cursor of the other party on the screen of the workstation of the user can be realized.
In order to display the other person's face on his / her workstation as a video image with a sense of reality, a high-performance encoding / decoding device and a high-speed and large-capacity communication path are required. That is,
In any case, it is difficult to display a realistic display on a communication line whose transmission amount is limited.

The present invention has been made in view of such a situation, and is intended to display images with a realistic display even when using a communication path with a limited transmission amount.

[0010]

In order to solve such a problem, the invention according to claim 1 is directed to image information representing a plurality of states.
Sent via a communication path with the video storage means for storing the
Status signal to a control signal that specifies the video that represents that status.
The state signal detecting means to be converted, and the
From the output of the frame number and the status signal detection means.
A display frame calculating means for determining an image frame number;
Based on the frame number calculated by the
Video reproduction means for reproducing the video stored in the video storage means
And a video displaying the video video reproduced by the video reproducing means.
Image display means . The invention of claim 2 is
According to the first aspect of the present invention, the displayed image is a control signal.
From the video stored in the video storage means based on the
Select a set of frame numbers and display the set
Are obtained by continuous reproduction . The invention of claim 3 is the invention according to claim 2,
The displayed video does not receive a signal from the communication
The frame number specified in advance
Or output a set of frame numbers to the video playback
It is obtained by doing. Claim 4
The invention of claim 1 is related to any one of claims 1 to 3.
And character / graphics reproducing means for generating at least one of characters and graphics in accordance with a control signal, wherein the video display means outputs the output of the character / graphics reproducing means to a video reproduced by the video reproducing means. This is displayed in a superimposed manner.

[0011]

According to the first and second aspects of the present invention, the video information stored on the receiving side is read out and displayed according to the status signal sent from the transmitting side. According to a third aspect of the present invention, the display content is determined by referring to a table in which the display content is written corresponding to the status signal. According to a fourth aspect of the present invention, in the first or second aspect of the invention, the display content determined according to the control signal is determined by performing a predetermined operation based on the control signal. The invention of claim 5 is the invention of claim 1 or claim 2.
According to the invention, the displayed image is captured in advance or created by computer graphics. According to a sixth aspect of the present invention, the frame number of the video to be displayed next is obtained from the frame number of the video displayed at a certain time, and the video stored in the video storage means is reproduced based on the frame number. It is. According to a seventh aspect of the present invention, in any one of the first to sixth aspects, an appropriate set of frame numbers is selected from the stored videos, and the set is continuously reproduced. According to an eighth aspect of the present invention, a frame number or a set of frame numbers specified in advance is displayed when there is no input for a predetermined time or more from the communication channel. According to a ninth aspect of the present invention, in any one of the first to eighth aspects, the output of the character / graphics is superimposed on the displayed video.

[0012]

FIG. 1 is a block diagram showing an embodiment of an apparatus according to the present invention. An interface 1 receives a signal transmitted through a wireless or wired communication path. Is commercially available as a modem or an interface board, and has functions such as procedural control or error correction for responding to signals transmitted at various speeds.

The state signal detecting means 2 includes an interface 1
Has the function of converting the status signal received from the communication path via the communication path and indicating the status of the other party into a control signal for selecting an image to be displayed on the receiving side. Since both the output of the interface 1 and the output of the state signal detecting means 2 are digital signals, a digital processing device such as a microprocessor can be used. The same applies to the sign frame calculating means 3 to which the control signal is supplied. The display frame calculating means 3 has a function of converting the control signal output from the state signal detecting means 2 into a frame number or a set of frame numbers of video images to be displayed. The image storage means 4 is means for storing video images, and can use a hard disk, a video tape recorder, a laser disk, a semiconductor memory, a CD-ROM, or the like. A plurality of these devices may be used in parallel.

The image reproducing means 5 includes the sign frame calculating means 3
The video storage means 4 is controlled based on the output of the above, and a necessary one is selected and read out from the stored video, and a calculation function of a personal computer, a workstation, a microprocessor or the like can be used. The video storage means 4 is a video tape recorder and its playback device;
In the case where the apparatus has a reproducing function such as a laser disk and its reproducing apparatus, the image reproducing means 5 only needs to send a command for controlling those reproducing functions. It is connected to the means 4 and displays its output. If the video storage means 4 does not have a reproduction function such as a hard disk or a semiconductor memory, the video reproduction means 5 reads out the video data from those storage means, reproduces the data, and sends the result to the video display means 6.

A video tape recorder playback device that can be controlled from a personal computer includes, for example, NEC PV-
There is S98, and the manual states that necessary control can be performed by sending an instruction in ASCII code via an RS232C cable. For example, a Pioneer laser disk player LD-
V540, and its manual states that it can be controlled from the computer via an RS232C cable. For a system capable of reproducing and displaying video data stored on a hard disk or a semiconductor memory by controlling the video data from a computer, see, for example, Reference 7 “Information Processing Society 45th Annual Meeting (1992) 5B-0”.
1 Sakagami, Hamakawa, Calendar Book: AV Extension of HyperStation Project-oriented GUI Tool InterViews ". The image display means 6 is for displaying a video image reproduced by the image reproduction means 5 or the image storage means 4, and a display device such as a CRT monitor can be used.

The apparatus of the present invention controls and displays a video image stored in advance based on a signal transmitted from a communication path. For example, in the case of a conventional videophone, a video image of the state of the transmitting side is transmitted and the receiving side reproduces the video image. Side can display video images. For example, if the sender detects a state such as “moving his mouth”, “turns to the right”, or “smiles” from the sender, and only the information is transmitted, Searches and displays "moving mouth", "right-pointing video", and "laughing video" corresponding to the state of the transmitting device from the video images stored in advance on the receiving device. Like that. As in this example, the state on the transmitting side and the state on the receiving side do not need to be strictly semantically identical, and it is only necessary that they correspond. As another example, suppose that the transmitting side is connected to a certain device, and a signal indicating the operating state of the device is constantly transmitted to the receiving side. When the signal indicating the occurrence of the abnormality is not transmitted from the transmitting side, the display device of the receiving side always displays the video image of the laughing person, and when the signal indicating the occurrence of the abnormality is transmitted, the display of the receiving side is displayed. The display device may also display a video image of a shouting person. On the receiving side, the situation can be understood more intuitively than when the text "abnormality occurred" is displayed.

FIG. 2 is a flowchart showing a processing procedure when displaying a video image. In the following description, the image storage means 4 captures and records faces showing various states of a person on the transmission side of the communication path. It will be described as having been performed. Specifically, it is assumed that an example of a frame stored in the video storage unit 4 is as shown in FIG. In FIG. 3, v01 to v0
There are eight video sequences numbered 5, 5, v21, v31, and in each sequence the frames in the sequence f01 to f30 are numbered. Each of these frames is obtained by concatenating the frame numbers of the sequence “v02: f15”.
Let's say you have a frame number such as For example, the sequence v
01 stores a video sequence in which a human rotates his face from the center to the left.

In FIG. 2, it is determined in step 100 whether or not there is a signal transmitted from the communication channel. If there is a signal, the input signal is detected by the state signal detecting means 2 as shown in step 101, as shown in step 101. Is performed, and if there is no signal from the communication path, the process enters a waiting state. The transmitted signal is, for example, a signal indicating the operating state of a device connected to the other side of the communication path, or a signal indicating the direction of the person at the other side of the communication path and the working state. In some cases, any digital signal sequence may be used.
The converted signal indicates a video image to be displayed, and is a signal sequence that can be interpreted by the sign frame calculating means 3. For example, it is assumed that the operating state of a certain machine is transmitted from a transmission side via a communication path. Specifically, it is assumed that a numerical value indicating the load state is sent. The state signal detecting means 2 makes the video signal “laugh” or “normal” according to the magnitude of the numerical value indicating the state of the transmitting side.
It is assumed that the state is switched to the three states of “angry”. For example, when the load state of the machine is light, an image of a "laughing face" is output, when it is heavy, an image of an "angry face" is output, and in other cases, an image of a "normal face" is output. The output signal is for selecting a video signal. These correspondences are given in a table format, for example, as shown in FIG. This correspondence is not limited to a table, and may be given by a calculation formula, a program, or the like.

Next, as shown in step 102, a process of converting the currently displayed frame number and the output of the video image signal converting means 2 into the frame number to be displayed is performed by the sign frame calculating means 3, and the video image to be displayed is displayed. Are obtained. The currently displayed frame number is provided from the video image reproducing means 4 to the sign frame calculating means 3. now,
As the initial screen, the image display means 6 displays v01: f01 in FIG.
FIG. 6 illustrates a change state of the video image when the video image is displayed. At this time, the state signal detecting means 2
From the signal instructing the right direction of the screen (hereinafter "→"
) Is output. The frame number being displayed in the table in FIG. 4 is v01: f01, and when the input signal is “→”, the frame number to be displayed next is v01: f.
01 to v01: f30, which is output as a set of frame numbers. This is a series of images in which a person turns his face from the front to the left.

Next, in this state, a signal indicating the upward direction of the screen (hereinafter, this signal is indicated by "↑") is sent to the state signal detecting means 2.
Output from At this time, according to the rule of FIG. 4, when the frame number is v01: f30 and the input signal is “↑”, v02: f30,
v03: f30 is output. This is an image in which the face that first turned to the left faces first to the upper left, and then to the top. In this way, faces oriented in a direction corresponding to the output content of the state signal detecting means 2 are sequentially displayed. Even if the video is not stored in a series of states, for example, v01: f3
0, v02: f30, v03: f30, etc., it is possible to display the recorded images in combination as appropriate, so that the display state can have a considerable degree of freedom.

Next, based on the output of the sign frame calculating means 3, the picture reproducing means 5 controls the picture accumulating means 4 to display the video picture of the designated frame on the picture display means 6 as shown in step 103. Control. Thereafter, in order to perform processing according to the signal from the communication path again, step 1 is executed.
Return to 00. After the video reproducing means 5 starts displaying the frame of the designated number, the process may return to step 100 without waiting for the end of step 103.

FIG. 5 shows the details of step 103 for displaying the above-mentioned designated frame.
Stores one or more video sequences, which are sets of continuous video frames, and when a video frame is output from the video sequence, the video reproducing means 5 sequentially turns the video frames into a continuous state. Search and display. However, a series of video frames having consecutive numbers is displayed continuously without being searched. If it is determined in step 120 that the display has not been completed, it is checked in step 121 whether or not the frame numbers to be displayed are continuous.
If they are continuous, the continuous frame is displayed as shown in step 122. After the display, it is determined whether or not there is a designation of a frame having a continuous frame number (step 126).
And the display is terminated if there is no specification. Step 1
If it is determined in step 21 that a series of frame numbers to be displayed are not consecutive, the first frame designated in step 125 is displayed, and after the display, step 12 is executed.
In step 6, the next frame to be displayed is searched, and step 1
Return to 20. At this time, if the display is not completed, the same processing is performed again to display the next frame. Step 1
If it is determined in step 20 that the display has been completed, this processing is terminated. Whether or not the display is completed may be determined by, for example, presetting the number of frames to be displayed and counting down a preset value every time one frame is displayed.

The signal for controlling the video storage means 4 differs depending on the type of the video storage means 4 to be displayed. For example, when the video storage means 4 is the above-mentioned PV-S98, a designated frame number is searched. This is an instruction to display or an instruction to put the video tape recorder in a playback state. When the video storage unit 4 is the above-described LD-V540, the command is a command for searching for a designated address or a command for reproducing. Even when the video storage means 4 is a hard disk or a semiconductor memory, it is converted into a control signal for displaying a designated frame number or a set thereof.

FIG. 7 is a flowchart showing a processing procedure when displaying a video image by the method of claim 8.
The feature of this device is that when a signal does not come from a communication path for a certain period of time, a video image designated in advance is taken out from the image storage means 4 and reproduced and displayed. First, as shown in step 130, the process starts after clearing the time measurement. In step 131, it is determined whether or not there is a signal from a communication path. In step 132, it is checked whether or not a predetermined time has elapsed, that is, whether or not a signal has been received from a communication channel for a predetermined time or more.
If no signal is received for a certain period of time or more, as shown in step 133, the video reproduction means 105 retrieves a video video frame or a set of frames specified in advance from the video storage means 4 and displays it. For example, the setting is made so that a laughing video of frame v01: f01 is displayed when a predetermined time or more has elapsed. In step 132, if the predetermined time has not elapsed, a signal from the communication path is waited.

FIG. 8 is a block diagram showing the construction of the ninth embodiment. Character / graphics reproducing means 7 is added to that of FIG. The video is reproduced and displayed on the video display means 6 in addition to the video from the video reproduction means 5. The feature of this device is that characters and graphics synchronized with the video image are displayed over the video image. FIG. 9 is a flowchart showing the operation of the apparatus shown in FIG. 8. When there is a signal from the communication channel in step 100, the character /
The graphics reproducing means 7 generates characters and graphics data according to a predetermined algorithm.

In this case, the information is displayed so as to be superimposed on the displayed video image, and the algorithm is arbitrary. For example, the signal output from the state signal detecting means 2 via the communication path is output as it is. May be. The animation may be controlled based on the output signal. The video image reproduced by the image reproducing means 5 and the character / graphics data reproduced in step 150 are superimposed and displayed on the image display means 6. In this case, a function for superimposing and displaying the video image and the graphics data is required for the image display means 6, and a product having such a function is, for example, NEC VP150. A combination of such a product and a CRT display can be used as the video display means 6.

As the video of the video storage means 4 used in the above example, a video of the other party may be stored in advance, or a video created by computer processing may be stored in the video storage means 4. If necessary, other than those stored in the video storage means 4, those generated by computer processing based on the contents instructed from the transmitting side may be used. In this case, the video information is created after receiving an instruction, so there is a time delay. However, even in a good use, the memory capacity can be reduced, so that the economy is improved.

In the above example, the next frame to be displayed is shown in FIG.
Is obtained from the table shown in
It can also be obtained by calculation based on the reference in FIG.

For example, v0 which is an image in which a person faces the upper left
2: Assume that f30 is displayed. At this time, it is assumed that “↑” indicating the upward direction of the screen is given as the request signal. At this time, the next displayed frame number is v03: f
n. That is, the calculation may be performed by replacing the part “n” with the number corresponding to the frame number being displayed. The formula may be arbitrarily set so as to be suitable for the video to be displayed.

[0030]

As described above, according to the present invention, only the signal indicating the state is transmitted on the transmitting side, and the video information stored in advance is read and displayed according to the signal on the receiving side. Even if the information amount of the signal sent from the transmitting side is extremely small compared to the method of directly sending the image, the image can be displayed, so that a realistic display can be performed even when using a communication channel with a limited amount of information to be transmitted. It has the effect of being able to.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of one embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of FIG.

FIG. 3 is a diagram showing an example of video information to be stored.

FIG. 4 is a diagram showing an example of a table for determining information to be displayed.

FIG. 5 is a flowchart illustrating an example of a method of changing a displayed frame in FIG. 2;

FIG. 6 is a diagram showing a change state of a displayed image.

FIG. 7 is a flowchart showing an operation of the embodiment showing a display state when there is no input for a fixed time.

FIG. 8 is a block diagram illustrating an example of an apparatus that displays characters or graphics together with video.

9 is a flowchart showing the operation of the device shown in FIG.

FIG. 10 is a diagram showing another example of how to determine a frame to be displayed.

FIG. 11 is a block diagram illustrating an example of a conventional device.

FIG. 12 is a block diagram showing another example of the conventional device.

[Explanation of symbols]

1, 10 interface, 2 state signal detecting means,
3 display frame calculation means, 4 video storage means, 5 video playback means, 6 12 video display means, 7 character / graphics playback means, 11 decoding means, 12 display means.

──────────────────────────────────────────────────続 き Continued on front page (58) Field surveyed (Int.Cl. ⁶ , DB name) H04N 7/14

Claims (4)

(57) [Claims] 1. An image display device for displaying, on a receiving side, a state of a transmitting side transmitted by a communication path with a limited transmission amount as an image on a receiving side, image storing means for storing image information representing a plurality of states. The status signal sent via the communication path indicates the status
State signal detecting means for converting the image into a control signal specifying the image
And the frame number of the currently displayed video and the status signal detection means.
Display for finding the frame number of the next image to be displayed from the output
Frame calculating means, and a frame number calculated by the display frame calculating means.
A video for reproducing the video stored in the video storage
Image reproducing means, and an image displaying a video image reproduced by the image reproducing means.
An image display device comprising: an image display unit. 2. The image display device according to claim 1, wherein the displayed image is stored in an image storage means based on a control signal.
Select an appropriate set of frame numbers from the stacked video
And the video playback means continuously plays back the set.
An image display device characterized by being obtained by: 3. An image to be displayed according to claim 2, wherein a signal does not come from a communication channel for a predetermined time or more.
The frame number specified in advance
Or output a set of frame numbers to the video playback
An image display device characterized by being obtained by: 4. The method according to claim 1, wherein :
Depending on the control signal,
Character / graphics playback means to generate at least one
The video display means includes the text on the video reproduced by the video reproduction means.
A video display device characterized by displaying the output of a character / graphics reproducing means in a superimposed manner .