JP3369966B2 - REMOTE CONTROL DEVICE AND COMPUTER-READABLE MEDIUM RECORDING REMOTE CONTROL PROGRAM - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote operation device for remotely operating a control device, a measuring device and the like, and a computer-readable medium recording a remote operation program.

[0002]

2. Description of the Related Art Among control devices such as robots and production equipment, and various measuring devices such as distance sensors, position sensors, and temperature sensors, at least a part of the devices that can be operated by a command from a computer There is a target device). FIG. 1 shows an example of a device (computer) that operates an operation target device. In the figure, 10 is an operation target device, 20 is a computer, and these are the communication unit 101 of the operation target device 10 and the computer 20. It is directly connected via an output device (interface) 201.

The computer 20 comprises an external storage device 202, a keyboard 203, a pointing device (eg mouse) 204, a display 205, a speaker 206, a microphone 207, a CCD camera 208 and an operation processing section 210.

The operation processing unit 210 also transmits a command to the operation target device 10 by a command transmission processing unit 211.
A data reception processing unit 212 that receives data from the operation target device 10, and a data reception result display 205.
A display save processing unit 213 that displays the image on the screen and saves it in the external storage device 202, the keyboard 203, and the mouse 20.
4, processing input from the user interface such as the microphone 207, generating a command for operating the operation target device 10, and transmitting the command transmission processing unit 211 and the data reception processing unit 2
And a command processing unit 214 for controlling the control unit 12.

Conventionally, another computer (second computer) is connected to a computer (first computer) directly connected to the above-mentioned operation target device via a network, and the second target computer operates from this second computer. There is a remote control device for operating.

FIG. 2 shows an example of a conventional remote control device, in which the output screen of the first computer is output to the second computer by software processing or the like, and the first computer is operated by using the input information of the second computer. The figure shows an example in which a computer is operated. In the figure, the same components as those in FIG. That is, 10 is the operation target device,
20a is a first computer, 30 is a second computer, and 40 is a network.

The first computer 20a includes the input / output device 201, the external storage device 202, and the keyboard 20 described above.
3, mouse 204, display 205, speaker 20
6, a microphone 207, a CCD camera 208, a network interface 209, and a direct connection side operation processing unit 220. The direct connection side operation processing unit 220 includes a direct operation processing unit 230 and a remote-controlled user interface processing unit 221.

The direct operation processing unit 230 is a device to be operated 1
Direct operation command transmission processing unit 2 for transmitting a command to 0
31, a direct operation data reception processing unit 232 that receives data from the operation target device 10, a direct operation display saving processing unit 233 that displays the direct operation data reception result on the display 205 and saves it in the external storage device 202,
It processes an input from a user interface such as the keyboard 203, mouse 204, microphone 207, generates a transmission command for directly operating the operation target device 10, and controls the direct operation command transmission processing unit 231 and the direct operation data reception processing unit 232. And a direct operation command processing unit 234 for performing the operation.

Remotely operated user interface processing unit 2
21 operates the user interface on the input side of the first computer 20a based on the input information from the second computer 30, and transmits the display result of the first computer 20a to the second computer 30.

The second computer 30 includes a network interface 301, an external storage device 302, a keyboard 303, and a pointing device (eg mouse) 3
04, display 305, speaker 306, microphone 3
07, CCD camera 308 and remote connection side operation processing unit 3
It consists of 20.

Further, the remote connection side operation processing section 320 transmits the input information from the user interface such as the keyboard 303, the mouse 304, the microphone 307, etc. to the first computer 20a, and the display result received from the first computer 20a. Is provided on the display 305.

The first computer 20a and the second computer 30 have network interfaces 209, 3
01 and the network 40.

FIG. 3 is another example of the conventional remote control device, in which the input / output device of the first computer is virtually arranged as the input / output device of the second computer by software processing or the like. In the figure, the same components as those in FIGS. 1 and 2 are represented by the same reference numerals. That is, 10 is an operation target device, 20b is a first computer, 30a is a second computer, and 40 is a network.

The first computer 20b includes the input / output device 201, the external storage device 202, and the keyboard 20 described above.
3, mouse 204, display 205, speaker 20
6, the microphone 207, the CCD camera 208, the network interface 209, and the direct connection side operation processing unit 2
It consists of 20a.

The direct connection side operation processing section 220a converts input information in a fixed format from the network interface 209 into output information to the input / output device 201, and
The remote-controlled bypass unit 222 is provided which converts input information in a fixed format from the input / output device 201 into output information to the network interface 209.

The second computer 30a includes the above-mentioned network interface 301 and external storage device 30.
2, keyboard 303, mouse 304, display 3
05, speaker 306, microphone 307, CCD camera 3
The remote connection side operation processing unit 320a is also included. The remote-connection-side operation processing section 320a includes an operation processing section 310 and a remote operation bypass section 322.

The operation processing section 310 includes a command transmission processing section 311 for transmitting a command to the operation target apparatus 10 and a data reception processing section 3 for receiving data from the operation target apparatus 10.
12, a display saving processing unit 313 that displays the data reception result on the display 305 and saves it in the external storage device 302, and processes input from the user interface such as the keyboard 303, the mouse 304, and the microphone 307.
A command processing unit 314 that generates a command for operating the operation target device 10 and controls the command transmission processing unit 311 and the data reception processing unit 312 is provided.

The remote control bypass unit 322 converts the transmission command from the command transmission processing unit 311 into a fixed format, transmits it to the first computer 20b via the network interface 301, and transmits it from the first computer 20b. The format input information is converted and transmitted to the data reception processing unit 312.

The first computer 20b and the second computer 30a include a network interface 209,
It is connected via 301 and the network 40.

[0020]

However, in the configuration of FIG. 2, the first computer is operated while the second computer is operating.
Second user interface on the computer's input side
Since the first computer is directly affected by the input information from the computer, there is a problem that the first computer becomes extremely difficult to operate other auxiliary operations or other programs.

Further, in the configuration of FIG. 3, in the case where the operation target device is set to detect an error when no input is made from the computer to the operation target device within a fixed time with respect to the output from the operation target device, the network is used. Through the first and second
There was a problem that normal operation could not be performed due to the delay in communication between computers.

An object of the present invention is to prevent an auxiliary operation or the like in a computer directly connected to an operation target device from being hindered even when an operation is performed from a computer connected to the network, and further, in a network. An object of the present invention is to provide a remote control device in which normal operation cannot be performed due to communication delay.

[0023]

In order to achieve the above-mentioned object, the present invention is a device for operating an operation target device capable of at least a part of operations by a command from a computer, which is provided through an input / output interface. In a remote control device including a first computer directly connected to a communication unit of an operation target device and a second computer network-connected to the first computer, the first computer is connected to the operation target device. Direct operation command transmission processing unit for sending commands, direct operation data reception processing unit for receiving data from the operation target device, direct operation display for displaying the direct operation data reception result on the display unit and saving it in the external storage device The send command that processes input from the save processing unit and the user interface and directly operates the operation target device Forms a direct operation processing unit having a direct operation command processing unit for controlling the direct manipulation command transmission processing unit and direct operation data reception processing unit, performed within a predetermined time in the operation target device from the second computer
A remote operation command receiving unit for receiving a macro command corresponding to a series of operations to be a remote control data transmission unit which transmits the operation result data to a second computer, by the macro command from the second computer A remote operation display processing unit that displays the operation result on the display unit,
Converts the received said macro command corresponding to a predetermined command or command sequence to direct operation command transmission processing unit comprises a remote macro operation processing unit for controlling the direct manipulation command transmission processing unit and the direct manipulation data receiver An operation that processes the input from the remote operation relay processing unit and the user interface and switches the control of the direct operation command transmission processing unit and the direct operation data reception processing unit to either the direct operation command processing unit or the remote macro operation processing unit. and a switching unit is provided, on the second computer, and a remote operation command transmitting section that transmits the macro command to the first computer, the remote control data reception processing unit for receiving the operation result data from the first computer processes the input from the user interface, and generates the macro command, the far A remote operation transmission command processing unit that controls the operation command transmission processing unit and the remote operation data reception processing unit, and a remote operation result display storage processing unit that displays the data reception result on the display unit and stores it in an external storage device. It is characterized in that a remote control processing unit is provided.

According to the present invention, while the operation is performed by the second computer, the user interface on the input side such as the pointing device of the first computer is changed to the second user interface.
Since it is not directly affected by the input device of the computer, it is possible to solve the problem that other auxiliary operation of the first computer or operation of another program becomes extremely difficult.

When the operation target device does not input to the operation target device within a fixed time with respect to the output from the operation target device, if the operation target device is set to detect an error, the first computer Since it is possible to respond, it is possible to solve the problem that normal operation may not be performed due to the influence of communication delay in the network.

[0026]

FIG. 4 shows an example of an embodiment of a remote control device of the present invention. In the figure, the same components as those of the conventional example are represented by the same reference numerals. That is, 10 is a device to be operated, 20c is a first (direct connection side) computer, 30
Reference numeral b is a second (remote connection side) computer, and 40 is a network.

The first computer 20c includes the input / output device 201, the external storage device 202, and the keyboard 20 described above.
3, a pointing device (eg mouse) 204,
Display 205, speaker 206, microphone 207,
CCD camera 208, network interface 20
9 and a direct connection side operation processing unit 220b.

The direct connection side operation processing unit 220b includes the above-described direct operation processing unit 230 and remote operation relay processing unit 240.
An operation switching processing unit 251 and a video / audio communication processing unit 25
2 and 2, which can mutually transmit and receive data. Further, the remote operation relay processing unit 240 includes a remote operation command reception processing unit 241 and a remote operation data transmission processing unit 2.
42, a remote operation display processing unit 243, and a remote macro operation processing unit 244.

The second computer 30b includes the above-mentioned network interface 301 and external storage device 30.
2, keyboard 303, pointing device (eg mouse) 304, display 305, speaker 30
6, a microphone 307, a CCD camera 308, and a remote connection side operation processing section 320b.

The remote connection side operation processing section 320b includes a remote operation processing section 330 and a video / audio communication processing section 341. Further, the remote operation processing unit 330 includes a remote operation command transmission processing unit 331 and a remote operation data reception processing unit 3
32, a remote operation transmission command processing unit 333, and a remote operation result display storage processing unit 334, which can mutually transmit and receive data.

The first computer 20c and the second computer 30b include a network interface 209,
It is connected via 301 and the network 40.

Here, the direct connection side operation processing unit 220b
Can be realized by a program (software) recorded in a computer-readable medium, and has another configuration (201 to 209) in the first computer 20c.
Can be controlled from the direct connection side operation processing unit 220b via the application programming interface of the basic software of the first computer 20c. Further, the remote-connection-side operation processing unit 320b can also be realized by a program (software) recorded in a computer-readable medium, and the other configurations (301 to 308) in the second computer 30b are remote-connection-side operation. Processing unit 33
It can be controlled from 0 to the second computer 30b through the application programming interface of the basic software.

The details of each unit will be described below together with the operation thereof.

In the direct operation processing unit 230, the direct operation command transmission processing unit 231 receives the operation target device 1 via the input / output device 201 and the communication unit 101 of the operation target device 10.
The operation target device 10 is operated by transmitting a predetermined command to 0. The data from the operation target device 10 is directly received by the operation data reception processing unit 232 via the communication unit 101 and the input / output device 201 of the operation target device 10.

The direct operation display storage processing section 233 performs processing for displaying the data received by the direct operation data reception processing section 232 on the display 205 or storing it in the external storage device 202. The direct operation command processing unit 234
It processes an input from a user interface such as the keyboard 203, mouse 204, microphone 207, generates a predetermined transmission command to be directly operated, and controls the direct operation command transmission processing unit 231 and the direct operation data reception processing unit 232.

In the remote operation relay processing unit 240, the remote operation command reception processing unit 241 receives a macro command for operating the operation target device 10 from the second computer 30b. Remote operation data transmission processing unit 242
Transmits the operation result data to the second computer 30b. The remote operation display processing unit 243 displays the operation result data by the macro command from the second computer 30b on the display 205.

The remote macro operation processing unit 244 converts the received macro command into a predetermined command or command string corresponding to the direct operation command transmission processing unit 231 based on the information stored in the external storage device 202, The direct operation command transmission processing unit 231 and the direct operation data reception processing unit 232 are controlled to obtain operation result data.

The operation switching processing unit 251 uses the first computer 20c only before and after the start of the remote operation so that the operation target device 1 can be operated.
When a direct operation for confirming the connection with 0 is performed, the remote operation and the direct operation can be easily switched. Input from the user interface such as the keyboard 203, the mouse 204, and the microphone 207 is performed. The direct operation command transmission processing unit 231 and the direct operation data reception processing unit 232 are controlled by the direct operation command processing unit 23.
4 or control from the remote macro operation processing unit 244.

The video / audio communication processing section 252 transmits the input information from the microphone 207 and the CCD camera 208 to the second computer 30b by the input from the user interface such as the keyboard 203, the mouse 204, the microphone 207, and the like. The second computer 30b receives information from the video / audio communication processing unit 341 and outputs the information to the display 205 and the speaker 206.

In the remote control processing section 330, the remote control command transmission processing section 331 is the first computer 20c.
Send a macro command to. The remote operation data reception processing unit 332 receives the operation result data from the first computer 20c.

The remote operation transmission command processing unit 333 processes the input from the user interface such as the keyboard 303, the mouse 304, the microphone 307, and the external storage device 30.
The macro command is generated based on the information recorded in 2 and the remote control command transmission processing unit 331 and the remote control data reception processing unit 332 are controlled.

At this time, if no input is made from the computer to the operation target device 10 within a fixed time with respect to the output from the operation target device 10, even if the operation target device 10 is set to detect an error, error detection is performed. By making a series of operations related to
Direct operation processing unit 2 without communication between c and 30b
Since the 30 and the remote operation relay processing unit 240 enable a series of operations related to error detection with the operation target device 10 only by the processing of the first computer 20c, the normal operation is not affected by the delay of the network 40. Operation becomes possible.

To describe the above-mentioned point in detail, in the case where the operation target device 10 is set to detect an error when no input is made from the computer to the operation target device 10 within a fixed time with respect to the output from the operation target device 10. in the output of the operation target apparatus 10 obtains the remote control macro processor 244 through a direct operation data reception processing unit 232, this time,
The remote operation macro processing unit 244 does not send the output to the second computer 30b, but corresponds to the direct operation command transmission processing unit 231 based on the information stored in the external storage device 202 as the macro command corresponding to this output. A predetermined command or a command string is converted, the command or the like is directly transmitted to the operation target device 10 via the operation command transmission processing unit 231, and after the transmission result is obtained, this is sent to the second computer 30b. Has become.

Also, the remote operation transmission command processing unit 333.
Is a direct operation processing unit 230 or a remote operation relay processing unit 240.
Instructing to change the macro command at the time of starting the device, the command sequence converted from the macro command, and the like.

The remote operation result display storage processing section 334 displays the received operation result data on the display 305 or stores it in the external storage device 302.

The video / audio communication processing unit 341 transmits the input information from the microphone 307 and the CCD camera 308 to the first computer 20c by the input from the user interface such as the keyboard 303, the mouse 304, the microphone 307, and the like. The information from the video / audio communication processing unit 252 of the first computer 20c is received and output to the display 305 and the speaker 306.

The video / audio communication processing units 252 and 34
In the first example, the remote operation is performed more efficiently by sharing the video / audio information, for example, the image of the operation target device 10 is captured by the CCD camera 208 of the first computer 20c, and the operation sound is recorded by the microphone 207 of the first computer 20c. To collect the sound
By sending these pieces of information to the second computer 30b, the operator on the second computer 30b side can easily know the operating status of the operation target device 10.

[0048]

As described above, according to the present invention,
A problem that it becomes extremely difficult to perform an auxiliary operation in a computer directly connected to the operation target device, and an operation is performed when the computer does not input to the operation target device within a certain time with respect to the output from the operation target device. With the setting in which the target device detects an error, it is possible to realize a remote operation device that can solve the problem that normal operation may not be performed due to the influence of communication delay in the network.

[Brief description of drawings]

FIG. 1 is a configuration diagram illustrating an example of a device that operates an operation target device.

FIG. 2 is a configuration diagram showing an example of a conventional remote control device.

FIG. 3 is a configuration diagram showing another example of a conventional remote control device.

FIG. 4 is a configuration diagram showing an example of an embodiment of a remote control device of the present invention.

[Explanation of symbols]

10: operation target device, 101: communication unit, 20c: first
(Direct connection side) computer, 30b: Second (remote connection side) computer, 40: Network, 201:
Input / output device, 202, 302: external storage device, 203,
303: keyboard, 204, 304: pointing device, 205, 305: display, 206, 3
06: speaker, 207, 307: microphone, 208, 3
08: CCD camera, 209, 301: network interface, 220b: direct connection side operation processing unit, 23
0: direct operation processing unit, 231: direct operation command transmission processing unit, 232: direct operation data reception processing unit, 233: direct operation display saving processing unit, 234: direct operation command processing unit, 240: remote operation relay processing unit, 241: Remote operation command reception processing unit, 242: Remote operation data transmission processing unit, 243: Remote operation display processing unit, 244: Remote macro operation processing unit, 251: Operation switching processing unit, 252, 34
1: video / audio communication processing unit, 330: remote control processing unit, 3
31: remote operation command transmission processing unit, 332: remote operation data reception processing unit, 333: remote operation transmission command processing unit, 334: remote operation result display storage processing unit.

─────────────────────────────────────────────────── ─── Continuation of front page (72) Inventor Takao Kakizaki 3-19-2 Nishishinjuku, Shinjuku-ku, Tokyo Nihon Telegraph and Telephone Corporation (56) References JP-A-62-48145 (JP, A) Kaihei 6-14377 (JP, A) JP 7-87576 (JP, A) JP 2-226305 (JP, A) JP 6-14378 (JP, A) JP 62-105595 ( (58) Fields investigated (Int.Cl. ⁷ , DB name) H04L 12/28 H04L 12/44-12/46 H04Q 9/00-9/16 H03J 9/00-9/06

Claims (2)

(57) [Claims] 1. A device for operating an operation target device capable of at least a part of operation by a command from a computer, the first computer being directly connected to a communication unit of the operation target device via an input / output interface. A direct operation command transmission processing unit for transmitting a command to the operation target device to the first computer, and a remote operation device including a second computer network-connected to the first computer; Direct operation data reception processing unit that receives data from the user interface, a direct operation display saving processing unit that displays the direct operation data reception result on the display unit and saves it in the external storage device, processes the input from the user interface, and operates Generates a send command that directly operates the target device, and sends the direct operation command transmission processing unit and the direct operation data. Direct operation processing unit having a direct operation command processing unit that controls the reception processing unit, when a constant in the operation target device from the second computer
A remote operation command reception processing unit that receives a macro command corresponding to a series of operations that should be performed within a period, a remote operation data transmission processing unit that sends operation result data to a second computer, and predetermined command or frame corresponding operation result and the remote operation display unit to be displayed on the display unit, and to direct the operation command transmitting section the macro command received by the macro command
The remote operation relay processing unit, which has a remote macro operation processing unit for converting the direct operation command transmission processing unit and the direct operation data reception processing unit, and the input from the user interface to process the direct operation command. the control of the transmission processing unit and direct operation data reception processing unit, and an operation switch unit for switching to either a direct operation command processor or remote macro operation processor provided in the second computer, said to the first computer and a remote operation command transmission processing unit for transmitting a macro command, the remote operation data reception processing unit for receiving the operation result data from the first computer, to process the input from the user interface, the Ma <br/> black command A remote operation transmission command processing unit that generates and controls the remote operation command transmission processing unit and the remote operation data reception processing unit. Parts and, remote control device, characterized in that a remote control unit comprising a remote operation result display storage processing unit for storing the external Symbol billion devices and displays on the display unit the data reception result. 2. A device for operating at least a part of an operation target device by a command from a computer, the first computer being directly connected to a communication unit of the operation target device via an input / output interface. And a computer-readable medium recording a remote control program for realizing a remote control device comprising a second computer network-connected to the first computer, wherein the program is the first or second computer. And a direct operation data reception processing unit for receiving data from the operation target device, and a direct operation data reception process unit for receiving data from the operation target device on the first computer. Direct operation display save processing unit that displays the result on the display unit and saves it in the external storage device And a direct operation command processing unit that processes an input from the user interface, generates a transmission command that directly operates the operation target device, and controls the direct operation command transmission processing unit and the direct operation data reception processing unit. At a fixed time in the operation processing unit and the operation target device from the second computer
A remote operation command reception processing unit that receives a macro command corresponding to a series of operations that should be performed within a period, a remote operation data transmission processing unit that sends operation result data to a second computer, and predetermined command or frame corresponding operation result and the remote operation display unit to be displayed on the display unit, and to direct the operation command transmitting section the macro command received by the macro command
The remote operation relay processing unit, which has a remote macro operation processing unit for converting the direct operation command transmission processing unit and the direct operation data reception processing unit, and the input from the user interface to process the direct operation command. An operation switching processing unit that switches the control of the transmission processing unit and the direct operation data reception processing unit to either the direct operation command processing unit or the remote macro operation processing unit is realized, and the first computer on the second computer. a remote operation command transmitting section that transmits the macro command to the remote control data reception processing unit for receiving the operation result data from the first computer, processes input from a user interface, said Ma <br/> black A remote operation transmission command that generates a command and controls the remote operation command transmission processing unit and the remote operation data reception processing unit. A remote operation program characterized by realizing a remote operation processing unit including a processing unit and a remote operation result display / save processing unit that displays a data reception result on a display unit and saves it in an external storage device is recorded. Computer readable medium.