JP7704839B2 - Robot system including a robot having a display unit - Google Patents

Description

The present invention relates to a robot system having a robot with a display unit.

The robot system includes a robot and a work tool attached to the robot. The robot can perform a predetermined task while changing its position and posture. The robot is composed of multiple components such as an arm. In the prior art, a robot system is known in which a display device is attached to a component of the robot's main body. For example, JP2018-529488A proposes a holding device for medical use in which a display unit composed of a ring of LED (Light Emitting Diode) members or the like is arranged on a joint axis, and the display unit indicates the direction of movement, etc.

Furthermore, in the prior art, cameras are known that can obtain a slant-free image even when the image is captured while the camera is tilted (for example, JP 4-81081 A).

Special table 2018-529488 publication Japanese Patent Application Publication No. 4-81081

The position and posture of a robot changes in order to move a work tool. If a display unit is placed on a component of the robot, when the robot is driven, the position and posture of the display unit changes along with the component. As a result, the orientation of the image displayed on the display unit changes relative to the worker, which may make it difficult for the worker to see the image displayed on the display unit.

A robot system according to a first aspect of the present disclosure includes a robot including a plurality of joint axes and a display unit disposed on a main body of the robot. The robot system includes a display control unit that controls an image displayed in a display area of the display unit, and a reference direction setting unit that sets a reference direction that is a direction used as a reference when determining the orientation of the image displayed on the display unit. The robot system includes a display coordinate system that is a coordinate system set for the display area of the display unit, a reference plane that is a reference plane of the display area of the display unit and is set on the display coordinate system, and a coordinate system setting unit that sets a reference point that is a point used as a reference for display on the display area of the display unit and is set on the display coordinate system. The robot system includes a coordinate system calculation unit that calculates a position and orientation of the display coordinate system based on the axis positions of each joint axis of the robot, and a display phase calculation unit that calculates a display phase direction that is a direction used as a reference for the orientation of the display on the display area of the display unit by projecting the reference direction onto the reference plane on the display coordinate system. The display control unit controls and displays the orientation of the image on the display area of the display unit based on the display phase direction and the reference point calculated by the display phase calculation unit .
A robot system according to a second aspect of the present disclosure includes a robot including a plurality of joint axes and a display unit disposed on a main body of the robot. The robot system includes a display control unit that controls an image displayed in a display area of the display unit, and a reference direction setting unit that sets a reference direction that is a direction that is a reference when determining the orientation of the image displayed on the display unit. The robot system includes a display coordinate system that is a coordinate system set for the display area of the display unit, a reference plane that is a reference plane of the display area of the display unit and is set on the display coordinate system, and a coordinate system setting unit that sets a reference point that is a point that is a reference for display on the display area of the display unit and is set on the display coordinate system. The robot system includes a coordinate system calculation unit that calculates a position and a posture of the display coordinate system based on the axis positions of each joint axis of the robot. The robot system includes a display phase calculation unit that projects a reference direction onto the reference plane on the display coordinate system, and further rotates the display system by a predetermined offset angle around the reference point, thereby calculating a display phase direction that is a direction that is a reference for the orientation of the display on the display area of the display unit. The display control unit controls and displays the orientation of the image on the display area of the display unit based on the display phase direction and the reference point calculated by the display phase calculation unit.
A robot system according to a third aspect of the present disclosure includes a robot including a plurality of joint axes and a display unit disposed on a main body of the robot. The robot system includes a display control unit that controls an image displayed in a display area of the display unit, and a reference direction setting unit that sets a reference direction that is a direction used as a reference when determining the orientation of the image displayed on the display unit. The display control unit controls and displays the orientation of the image on the display area of the display unit so as to display the image in a predetermined posture relationship with the reference direction based on the axis positions and reference directions of each joint axis of the robot. The reference direction setting unit sets the reference direction based on a position designated by an operator on the display area of the display unit.

According to an aspect of the present disclosure, a robot system can be provided that includes a display unit that is attached to the main body of the robot, and that allows images displayed on the display unit to be easily viewed when the position and posture of the robot change.

1 is a schematic diagram of a robot system according to an embodiment. 2 is a schematic diagram illustrating a joint axis of a robot according to an embodiment. FIG. FIG. 1 is a block diagram of a robot system according to an embodiment. FIG. 2 is a schematic diagram of the robot illustrating a planar display unit attached to the robot. FIG. 2 is a schematic diagram of the robot illustrating a curved display unit attached to the robot. 1 is a first image displayed on a display unit. 11 is a second image displayed on the display unit. 11 is a third image displayed on the display unit. 5 is a flowchart of control for displaying an image on a display unit in the embodiment. FIG. 2 is a plan view illustrating a display coordinate system, a reference point, a reference plane, and a display phase direction. FIG. 11 is a schematic diagram illustrating a first control for setting a reference direction. FIG. 11 is a schematic diagram illustrating a second control for setting the reference direction. FIG. 13 is a first schematic diagram of the robot illustrating a third control for setting a reference direction. FIG. 13 is a second schematic diagram of the robot illustrating the third control for setting the reference direction. FIG. 13 is a third schematic diagram of the robot illustrating a third control for setting a reference direction. FIG. 13 is a schematic diagram illustrating a fourth control for setting a reference direction. FIG. 13 is a schematic diagram illustrating a fifth control for setting a reference direction. FIG. 11 is a schematic diagram illustrating a first control for setting a display phase direction. FIG. 11 is a schematic diagram illustrating a second control for setting a display phase direction. FIG. 13 is a schematic diagram of a robot illustrating a third control for setting a display phase direction. FIG. 13 is a schematic diagram illustrating a third control for setting a display phase direction. FIG. 13 is a schematic diagram illustrating a fourth control for setting a display phase direction. 1 is a schematic diagram showing a display phase direction of a display unit when the robot is in a first position and posture. FIG. 11 is a schematic diagram showing a display phase direction when the robot is driven from a first position and posture and display control is performed. FIG. 11 is a schematic diagram showing a display phase direction when the robot is driven from a first position and posture and no display control is performed. FIG. FIG. 2 is a schematic diagram of the robot in a second position and orientation. 11 is a schematic plan view showing the display phase direction of the display unit when the robot is in a second position and posture. FIG. 13 is a schematic plan view showing a display phase direction when the robot is driven from a second position and posture and display control is performed. FIG. 13 is a schematic plan view showing the display phase direction when the robot is driven from a second position and posture and display control is not performed. FIG. FIG. 13 is a schematic diagram of the robot in a third position and orientation. 13 is a schematic front view showing the display phase direction of the display unit when the robot is driven from a third position and posture and display control is performed. FIG. 13 is a schematic front view showing the display phase direction of the display unit when the robot is driven from a third position and posture and the display is not controlled. FIG. 4 is a fourth image displayed on the display unit. 13 is a fifth image displayed on the display unit. 6 is a sixth image displayed on the display unit. 7 is a seventh image displayed on the display unit. 8 is an eighth image displayed on the display unit. This is a ninth image displayed on the display unit. This is the tenth image displayed on the display unit. This is an eleventh image displayed on the display unit.

A robot system according to an embodiment will be described with reference to Figures 1 to 40. The robot system according to the present embodiment includes a robot including a plurality of joint axes and a display device that displays predetermined information. The display unit of the display device is disposed in the main body of the robot.

(Robot System)
1 shows a schematic diagram of a first robot system in this embodiment. A robot system 6 in this embodiment has a function of transporting a workpiece. The robot system 6 is equipped with a robot device including a hand 2 as a work tool (end effector) and a robot 1 that changes the position and posture of the hand 2. The robot system 6 is equipped with a control device 4 that controls the robot 1 and the hand 2.

The robot 1 of this embodiment includes a base unit 14 fixed to an installation surface and a swivel base 13 supported by the base unit 14. The swivel base 13 is formed to rotate relative to the base unit 14. The robot 1 includes an upper arm 11 and a lower arm 12. The lower arm 12 is rotatably supported by the swivel base 13 via a joint unit. The upper arm 11 is rotatably supported by the lower arm 12 via a joint unit. The upper arm 11 also rotates around a rotation axis parallel to the extension direction of the upper arm 11. The robot 1 includes a wrist 15 connected to an end of the upper arm 11. The wrist 15 is rotatably supported by the upper arm 11 via a joint unit. The wrist 15 includes a flange 16 that rotates around a rotation axis along the extension direction of the wrist 15. The hand 2 is fixed to the flange 16.

Figure 2 shows a schematic diagram of the robot illustrating the joint axis of the robot of this embodiment. In this invention, a joint axis is an axis of a joint part that connects links constituting a robot, and is a part that changes the positional or angular relationship between the links. By changing the axis position of each axis (the position is the angle in the case of a rotation axis, and the length of displacement in the case of a linear axis), the positional relationship between the links can be changed. As a result, at least one of the position and posture of the tip of the robot can be changed. Note that an actuator for moving the axis position of the joint axis may be installed at a location other than the part that becomes the joint axis.

1 and 2, the robot 1 of this embodiment is configured with six joint axes. These six joint axes are, in order from the side of the pedestal 59 which is the base portion 14 of the robot 1, the first joint axis is the J1 axis 51, the second joint axis is the J2 axis 52, the third joint axis is the J3 axis 53, the fourth joint axis is the J4 axis 54, the fifth joint axis is the J5 axis 55, and the sixth joint axis is the J6 axis 56.

2, the J1 axis 51, J4 axis 54, and J6 axis 56 are configured as a rotation axis R1 that rotates around the links connecting the joint axes. The J2 axis 52, J3 axis 53, and J5 axis 55 are configured as a rotation axis R2 that rotates around a direction perpendicular to the links connecting the axes.

In this embodiment, the hand 2 grips and releases the workpiece. The hand 2 grips the workpiece by closing opposing claws. The work tool is not limited to a hand that grips the workpiece. Any work tool can be attached to the robot depending on the work to be performed by the robot system. For example, if the robot system performs arc welding, a welding torch can be attached to the robot.

Figure 2 is a simplified explanatory diagram showing the configuration of the axes of the robot 1. If the origin of the joint axis is the origin of the coordinate system (joint coordinate system) set for each axis and also the point where the links are connected, the position of the origin of the axis is expressed as a position on the coordinate system set in space. In the following, the coordinate system set in space and used as a reference when expressing the position and posture of the robot, such as the robot's joint axis, links, tip, and work tool attached to the tip, is referred to as the robot's reference coordinate system 87. The reference coordinate system 87 is a coordinate system in which the position of the origin and the orientation of the coordinate axes are fixed to the space in which the robot is placed.

In this embodiment, when the joint axis is a rotation axis, the position of the joint axis is the angle of the rotation axis. Moving the position of the joint axis means changing the position by rotating the rotation axis. When the joint axis is a linear axis, the position of the joint axis is the position in the movement direction of the linear axis. Moving the position of the joint axis means changing the position by moving the linear axis. When referring to the position of the origin of the joint axis, it refers to the position of the origin of the coordinate system set for each axis on the coordinate system set for the space. The coordinate system set for the space is a coordinate system for expressing at least one of the position and posture of the tip (work tool) of the robot 1, or the flange 16 for attaching the tip to the robot 1, or the coordinate system set for each axis, on an orthogonal coordinate system fixed for the space.

In addition, the coordinate system set for the robot device is set as the tool coordinate system to represent at least one of the position and posture of the robot 1 on the robot's reference coordinate system 87 set in space. The origin of the tool coordinate system, the point to be translated, and the center point when rotating are set as the control points. In this embodiment, all six axes of the robot 1 are rotational axes, but the robot 1 may also include a linear axis. In addition, the robot 1 in this embodiment is a vertical articulated robot composed of six axes. However, any robot having another form may be used as long as the position of each axis can be controlled and the orthogonal position can be controlled.

In addition, when the robot is mounted on a device that changes the position and posture of the robot, such as a running axis or another drive device, the robot's reference coordinate system may be set in space so that the position and posture of the robot or parts of the robot can be determined taking into account the drive of those drive devices.

Figure 3 shows a block diagram of the robot system in this embodiment. With reference to Figures 1 to 3, the robot 1 includes a robot driving unit that changes the position and posture of the robot 1. The robot driving unit includes a robot driving motor 19 as an actuator that drives components such as an arm. The robot driving unit drives actuators arranged on the joint shafts so as to move the rotational position of each joint shaft of the robot 1.

The robot driving unit may be configured based on any principle or power as long as it can drive and displace the joint axis of the robot. Furthermore, if the joint axis is a linear axis, the robot driving unit drives an actuator arranged on the joint axis to move the position on the linear axis. The hand 2 is equipped with a hand driving unit that drives the hand 2. The hand driving unit includes a pressure pump and a valve, etc., for driving the claws of the hand 2.

The control device 4 includes an arithmetic processing device (computer) having a CPU (Central Processing Unit) as a processor. The arithmetic processing device has a RAM (Random Access Memory) and a ROM (Read Only Memory), etc., connected to the CPU via a bus. The operation program 41 includes commands for driving the robot 1 and the hand 2. The robot system 6 transports the workpiece by driving based on the operation program 41.

The control device 4 includes a memory unit 42 that stores information. The memory unit 42 stores information related to the control of the robot 1 and the hand 2. The operation program 41 is stored in the memory unit 42. The memory unit 42 can be configured with a non-transitory storage medium. For example, the memory unit 42 can be configured with a storage medium capable of storing information, such as a volatile memory, a non-volatile memory, a magnetic storage medium, or an optical storage medium.

The control device 4 includes an operation command unit 43 that sends out operation commands. The operation command unit 43 corresponds to a processor that operates according to the operation program 41. The processor functions as the operation command unit 43 by reading the operation program 41 and implementing the control defined in the operation program 41. The robot drive unit is driven based on the operation command of the robot 1 output from the operation command unit 43, and the rotational position of the robot drive motor 19 in the joint axis of the robot 1 is changed.

The operation command unit 43 sends an operation command to the robot drive circuit 45 to drive the robot 1. The robot drive circuit 45 includes an electrical circuit that drives the robot drive motor 19. The robot drive circuit 45 supplies electricity to the robot drive motor 19 based on the operation command. The operation command unit 43 also sends an operation command to drive the hand 2 to the hand drive circuit 44. The hand drive circuit 44 includes an electrical circuit that drives the hand drive unit. The hand drive circuit 44 supplies electricity to the hand drive unit based on the operation command. The control device 4 in this embodiment is installed away from the robot 1, but is not limited to this form. The control device may be arranged inside the robot.

The robot 1 is equipped with a rotational position detector 18 that outputs axis position data, which is the rotational position of the joint axes of the robot 1. The rotational position detector 18 is composed of, for example, an encoder. The position and posture of the robot 1 can be detected by the output of the rotational position detector 18. In this embodiment, the rotational position detector 18 is attached to the robot drive motors 19 that are arranged corresponding to each joint axis.

The robot system 6 of this embodiment is equipped with a display device that displays any information. The display device includes at least one display unit 60 arranged on the main body of the robot 1. In FIG. 1, a display unit 61a is attached to the joint between the swivel base 13 and the lower arm 12 as an example of the display unit 60. The display device includes a processing unit 21 that generates images to be displayed on the display unit 60 and processes commands from the display unit 60. In this embodiment, the processing unit 21 is configured by the control device 4.

The processing unit 21 includes a reference direction setting unit 22 that sets a reference direction, which is a direction used as a reference when determining the orientation of an image displayed on the display unit 60. The processing unit 21 includes a coordinate system setting unit 23 that sets a display coordinate system, which is a coordinate system set for the display area of the display unit 60.

The processing unit 21 includes a coordinate system calculation unit 24 that calculates the position and orientation of a display coordinate system based on the axis position of each axis of each joint axis of the robot 1. The processing unit 21 includes a display phase calculation unit 25 that calculates a display phase direction that is a reference direction for the orientation of the display on the display area of the display unit 60. The processing unit 21 includes an offset angle setting unit 26 that sets the offset angle of the image displayed by the display unit. The processing unit 21 includes a command processing unit 29 that processes commands input by operating the display unit 60. The processing unit 21 includes a display control unit 27 that controls the image displayed in the display area of the display unit 60.

The processing unit 21 corresponds to a processor that operates according to the operation program 41. The processor functions as the processing unit 21 by reading the operation program 41 and implementing the control defined in the operation program 41. Furthermore, each of the units included in the processing unit 21, namely, the reference direction setting unit 22, the coordinate system setting unit 23, the coordinate system calculation unit 24, the display phase calculation unit 25, the offset angle setting unit 26, the display control unit 27, and the command processing unit 29, corresponds to a processor that operates according to the operation program 41. The processor functions as each unit by implementing the control defined in the operation program 41.

(Display)
The robot system 6 of this embodiment includes one or more display units 60 on the robot's main body, such as the links of the robot's arms, joint shafts, and arm tips. The display units 60 are attached to components of the robot that change their position and posture when driven by the robot drive unit. The display units 60 change their position and posture when the robot is driven by the joint shafts. The robot may also be mounted on a drive unit that changes its position and posture. The present invention is also applicable to cases where the position and posture of the display unit 60 change depending on the drive unit. The display unit 60 of the display device may be any display panel that can display characters or pictures so that they can be recognized. For example, a liquid crystal display panel or an organic EL (Electro Luminescence) display panel may be used as the display unit 60.

The display unit 60 is preferably thin and configured so that it can be appropriately attached to the part where it is to be installed. It is preferable that the display unit 60 has a shape that does not protrude significantly from the main body of the robot. Furthermore, when the display unit 60 is placed at the end of the robot, it is preferable that the display unit 60 has a shape that is integrated with the main body of the robot and has a smooth outer surface.

Figure 4 shows a schematic diagram of a robot illustrating various examples of display units in this embodiment. Display units 61a, 61b, 61c, 62, 63a, 63b, 64, and 65 are formed in a plate shape so as to have a planar display area. Display units 61a and 61b are fixed to the joint unit so that their display areas are perpendicular to the J2 axis 52 and the J3 axis 53. Display unit 61c is fixed to the joint unit so that their display area is perpendicular to the J5 axis 55. Display units 61a, 61b, and 61c move together with the joint unit.

Display unit 62 is fixed to the swivel base 13 so that its display area is parallel to the J1 axis. Display unit 62 rotates together with the swivel base 13. Display units 63a and 63b are fixed to the upper arm 11 so that their display area is parallel to the J4 axis 54. Display units 63a and 63b move together with the upper arm 11. Display unit 64 is arranged so that its display area is parallel to the J5 axis 55. Display unit 65 is fixed to the wrist 15 so that its display area is parallel to the J6 axis 56. Display units 64 and 65 move together with the wrist 15. In this way, display units 61a, 61b, 61c, 62, 63a, 63b, 64, and 65 are fixed to components of robot 1 and move together with the components of robot 1. Furthermore, each display unit rotates around the joint axis of robot 1 as the rotation axis.

Figure 5 shows a schematic diagram of another robot for explaining various examples of the display unit of this embodiment. The display units 66 and 67 are composed of flexible display panels. The display units 66 and 67 are formed so that the display area extends in the circumferential direction along the surface of the component of the robot 1. The display units 66 and 67 can be composed of a thin and flexible display panel such as an organic EL display panel. The display unit 66 is fixed so as to be wrapped around the lower arm 12. The display unit 66 is formed so that the display area is parallel to the extension direction of the lower arm 12. The display unit 67 is fixed so as to be wrapped around the upper arm 11. The display unit 67 is formed so that the display area is parallel to the extension direction of the upper arm 11. The display units 66 and 67 are fixed to the component of the robot 1 and move together with the component of the robot 1.

In this embodiment, the display units 66 and 67 are configured with one display panel, but are not limited to this form. Multiple display panels may be arranged around the components of the robot 1 to configure the display unit so as to surround the components. A single image may be displayed on multiple display panels.

FIG. 6 shows a first image displayed on the display unit. FIG. 7 shows a second image displayed on the display unit. In the following embodiment, the display unit 61a disposed between the swivel base 13 and the lower arm 12 will be described as an example among various display units. The display control unit 27 can display an image in the display area of the display unit 61a based on data transmitted from another device. Information displayed on the display unit includes, for example, the operating state of the robot, the name of the operating program being executed, the execution status of the operating program, the alarm status, the driving speed of the robot, the number of times the work has been executed, and the number of times removal has failed. Here, the driving speed or operating speed of the robot may be the translation speed or rotation speed of the tip of the robot or the tool coordinate system. Furthermore, the driving speed or operating speed of the robot may be the rotation speed of each joint axis, etc. The same applies in the following description unless otherwise specified.

The information to be displayed on the display unit includes the cause and countermeasures when a problem occurs, the content of the next task to be performed, the connection status of the network to which the robot is connected, the load status of the actuators included in the robot drive unit that drives the robot, the current time, date and time, and the elapsed time of the task, etc. It is preferable that the information to be displayed on the display unit is information useful to the worker by being displayed on the main body of the robot, such as information related to the robot's task or settings, or any information transmitted to the robot from an external device, etc.

The first image 71a displays information about whether the robot drive motor is running (State), the name of the operating program being executed (Prog.), the drive speed around the joint axis (Speed), the number of times the program has been executed (Count), the time, and the date. The second image 71b also displays information about the robot being driven.

Figure 8 shows a third image displayed on the display unit. The display unit may have not only a display function for displaying information, but also an input function for inputting predetermined information by the operator's operation. The display device can display an input image for inputting predetermined information on the display unit. The third image 72a is an input image for setting the drive speed around the joint axis of the joint axis on which the display unit is arranged or a desired joint axis. In addition, there are cases where joint axes other than one joint axis are driven together with one joint axis. In this case, when changing the drive speed around one joint axis, the drive speeds of all joint axes may be adjusted so that the position and posture of the tip of the robot do not change. In addition, the movement speed of the tip of the robot in the running operation program may be changed in the input image displayed on the display unit.

The display device may function as an input device in which information is input by an operator's operation in response to an input image. Furthermore, the display device may function as a teaching device for manually driving the robot or creating an operating program. For example, the display device may be provided with a touch panel type display panel as a display unit that allows input operations. Alternatively, the display device may be provided with an emergency stop button or the like for performing a desired function.

Thus, the display device in this embodiment includes both devices with only a display function and devices with a display function and an input function. With reference to FIG. 3, a command input to the display unit 60 of the display device is sent to the command processing unit 29. The command processing unit 29 processes the input command. For example, the command processing unit 29 sends a robot operation command to the operation command unit 43, or changes the setting values for driving the robot stored in the memory unit 42.

The display control unit 27 can display any input image in the display area. For example, it can display an image of a software keyboard, an image of numbers only, or an image in which a bar-shaped image such as a slide bar is moved to change the numerical value to be input. The display control unit 27 can also display an image in which a specific image is rotated to perform continuous input operations, an image that rotates like the dial of an analog telephone, or buttons related to changing the screen.

Any input method can be used for inputting information, such as inputting information by touching the display area as with a touch panel, or inputting information non-contact with a capacitance sensor. When inputting information non-contact, information can be input by operating in the space above the display area. As a device for detecting an input operation, a device using any sensing method, such as a capacitance sensor, an optical sensor, an optical camera, an infrared camera, or an ultrasonic sensor, can be used.

Furthermore, a contact or non-contact sensor may be operated as a contact sensor that detects contact with a person or surrounding objects, in addition to being an input device for inputting information. In other words, by switching the operating mode, a sensor that detects input operations may be used as a sensor for ensuring safety.

The information input to the display device is transmitted to another device, such as the robot's control device or a device connected via a network, via wired or wireless communication. Alternatively, the input information may be stored and remembered in a teaching device that includes a display function. Information calculated or processed based on the input information may be displayed on the display unit of the display device.

The input image, which is an image for the worker to input information, is preferably configured to allow input of any information for setting or checking. For example, the input image is preferably configured to allow input of changes in set values, changes in the robot's driving speed, instructions to put the robot on standby, changes in the robot's operation, driving the robot, or interruption of operation. Furthermore, the input image may be an image for inputting any information, such as input of information on tools or workpieces for replacing tools attached to the tip of the robot, setting a coordinate system, position teaching, changing to a direct teach mode, or changing to an automatic operation mode. It is preferable to limit the functions for which the set values can be manipulated to functions that are frequently used or simple functions. Alternatively, the input image may be formed so that the type of input function can be changed.

Furthermore, when inputting information into the display device, the display unit may display an image that accepts input of specific information to enable the worker to perform input operations. The display unit may switch to an image that enables the worker to perform input operations when the worker inputs specific information. For example, a button that enables input may be pressed for a predetermined time or longer, a password may be entered, or a touching finger may be moved so as to pass through a predetermined point on the display area. In this way, the display control unit may display an image that prohibits input operations to prevent erroneous input operations, etc. Then, an image that enables input operations when the worker performs an operation such as entering a password is displayed.

When inputting information on the display device, at least one of the authority to perform the input operation and the authority to display the information may be changed by a specified input operation. For example, by performing a specified input operation, it is possible to change the items that can be set, change the robot's behavior, change the range of values that can be set, or change the information that is displayed. In this way, the display device may change the instructions or information that can be input, or change the data or information that can be displayed.

The display unit may be formed so as to be detachable from the main body of the robot. In particular, the display device having the display unit and the display device constituting the teaching device may be formed so as to be detachable from the main body of the robot. In this case, the display device may be provided with a battery. Alternatively, it is preferable that the display device is supplied with power when attached to the main body of the robot. In addition, the device that transmits information to be displayed on the display device or the device that transmits information inputted to the display device may be formed so as to be connected by wire or wirelessly.

The display coordinate system set for the display unit can be configured to be a coordinate system that is generated in advance according to the mounting position of the display unit by mounting the display unit at a predetermined position on the robot's body. Alternatively, the display coordinate system may be directly input after mounting the display unit, or may be set by a conventional method.

When a display device detachable from a robot functions as a teaching device, it can be used only when the teaching device is needed. Or, when a robot is equipped with multiple display units, the teaching device can be installed and used only in the parts of the robot where it is needed. This makes it possible to reduce the cost of the robot system while improving the operability of the robot, and to reduce the risk of the teaching device being damaged by collision with peripheral devices while the robot is operating.

(Basic control to display images)
Fig. 9 shows a flow chart of control for displaying an image on the display unit in this embodiment. Referring to Fig. 3, before the display control unit 27 displays an image, the coordinate system setting unit 23 sets a display coordinate system corresponding to the display unit, and a reference plane and a reference point set in the display coordinate system. The coordinate system setting unit 23 sets the display coordinate system, the reference plane, and the reference point, for example, in response to an operation by an operator. Furthermore, the reference direction setting unit 22 sets a reference direction, which is a direction serving as a reference for display, by a predetermined control. The storage unit 42 stores the display coordinate system, the reference plane, the reference point, and the reference direction.

With reference to Figures 3 and 9, in step 111, the processing unit 21 acquires a display coordinate system, a reference plane, and a reference point from the memory unit 42. In step 112, the processing unit 21 acquires a reference direction from the memory unit 42.

In step 113, the coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system based on the position and orientation of the robot 1. Specifically, the coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system based on the output of the rotational position detector 18. The position and orientation of the reference plane expressed in the display coordinate system and the position of the reference point are calculated.

In step 114, the display phase calculation unit 25 calculates the display phase direction on the reference plane based on the reference direction and the reference point. That is, the display phase calculation unit 25 sets the orientation of the image on the display unit. In step 115, the display control unit 27 generates an image on the reference plane based on the display phase direction. Then, in step 116, the display control unit 27 displays an image on the display unit 60 based on the image on the reference plane. Next, these controls will be described in detail.

(Coordinate system setting section)
Fig. 10 shows a schematic plan view for explaining the display coordinate system, the reference plane, the reference point, and the display phase direction. Fig. 10 corresponds to a plan view of the display area of the display unit 61a. The coordinate system setting unit 23 of the processing unit 21 sets a display coordinate system 88, which is a coordinate system set for the display area of the display unit. The coordinate system setting unit 23 sets a reference plane 82, which is a reference plane for the display area of the display unit and is set on the display coordinate system, and a reference point 83, which is a point serving as a reference for display on the display area of the display unit and is set on the display coordinate system, for the display area of the display unit.

The display coordinate system can be set at any position and orientation relative to the display area of the display unit. The display coordinate system represents the position and orientation of the display unit or display area installed on the robot's main body on a coordinate system that serves as a reference, such as the robot's reference coordinate system. In the display unit 61a, the display coordinate system is set so that the plane formed by the two coordinate axes and the display area are on the same plane.

The reference plane is a plane that is used as a reference when representing an image to be output to the display area of the display unit on a plane. The reference plane is also a plane that is set on the display coordinate system. The position and orientation of the reference plane in the display coordinate system can be determined in advance. In the display unit 61a, the reference plane is set on a plane that includes two coordinate axes of the display coordinate system. The reference point is a point that determines the central position of the image to be output to the display area of the display unit. The orientation of the image is determined by rotating the image on the reference plane around the reference point as the center of rotation.

The reference direction can be set in any direction independent of the display coordinate system. The reference point is set on the reference plane. The reference plane can be set independent of the direction of the coordinate axes of the display coordinate system. For example, the reference plane may be inclined with respect to the XY plane of the display coordinate system. However, if the display area of the display unit is planar, it is preferable that the plane formed by the display area, the reference plane, and the two coordinate axes of the display coordinate system are coplanar. Setting in this manner can reduce the amount of calculations.

If the display area of the display unit is curved rather than flat, a reference point can be set on the display area, and the tangent plane at the reference point can be used as the reference plane. In this case, even if the XY plane and the reference plane in the display coordinate system are coplanar, the reference plane does not include the display area.

The display phase calculation unit 25 determines the display phase direction on the reference plane, so that the display control unit 27 can generate an image to be displayed on the reference plane. Next, the display control unit 27 projects the image on the reference plane onto the display area of the display unit, so as to generate an image to be displayed on the display unit.

The coordinate system setting unit 23 can set the display coordinate system 88, the reference plane 82, and the reference point 83 by arbitrary control. For example, the operator can predetermine the display coordinate system 88, the reference plane 82, and the reference point 83 based on the shape of the display area of the display unit, and input them to the processing unit 21. Basically, these setting values are determined for the display unit, and can be determined in advance as standard setting values before attaching the display unit to the robot. The operator can change the setting values as necessary. For example, after setting standard values for the display unit, the operator may change the display standard values to suit the work depending on the state of the robot to be displayed on the display unit.

The coordinate system setting unit 23 may also automatically set the display coordinate system 88, the reference plane 82, and the reference point 83 for the display unit. For example, the display unit may have a wide display area. As shown in the displays 66 and 67 of FIG. 5, a flexible display unit may be wrapped around a component of the robot. In such a case, the coordinate system setting unit 23 may automatically set the display coordinate system, the reference plane, and the reference point so that the image is displayed in the same direction regardless of the rotational position of the component. For example, the coordinate system setting unit 23 may automatically set the setting value to be displayed by moving it around the joint axis.

When forming the display units 66 and 67, multiple display units may be combined and arranged to surround the components of the robot. In this case, a display coordinate system, a reference plane, and a reference point may be set for each display area of each display unit. Alternatively, multiple display units may be combined to form a single display unit in a specified part of the robot, and a display coordinate system, a reference plane, and a reference point may be set. In this case, images may be distributed to each display unit and controlled to be displayed.

The display control unit 27 controls and displays the position and orientation (phase) of the display in the display area of the display unit based on a reference point 83, which is a reference point for the display on the display area of the display unit, and a display phase direction (direction indicated by arrow 84), which is calculated by the display phase calculation unit 25 and is a direction that is the reference for the orientation of the display on the display area of the display unit. For example, the display control unit 27 can display the image so that the direction indicated by arrow 84 is the top of the image.

In this embodiment, the display control unit 27 determines the center position of the display on the display area of the display unit using the reference point 83. The display control unit 27 determines the display orientation by rotating the image on the reference plane 82 with the reference point 83 as the center of rotation. In this embodiment, the origin of the display coordinate system 88 and the reference point are located at the same position, but they may be located at different positions.

The display phase direction indicated by arrow 84 is set to lie on reference plane 82 and pass through reference point 83. When the display area of the display unit is planar, it is preferable that reference plane 82 is parallel to the display area or on the same plane as the display area. In this embodiment, the display area of the display unit is planar, but it may have a shape other than planar, such as a curved shape.

When the display area of the display unit is curved, the tangent plane at a specified point on the display area can be set as the reference plane. A specified point on the display area can be set as the origin of the display coordinate system, and the reference plane can be a plane formed by two coordinate axes of the display coordinate system. The image displayed in the display area of the display unit can be displayed based on the reference point and the display phase direction, or an image on the reference plane can be projected onto the display area and displayed. The image displayed in the display area can be displayed in any manner, such as based on calculated information.

The display area of the display unit 61a (the portion that displays the image of the display panel) is circular, but may be any shape, such as a regular polygon. In this case, the reference point can be located in the center of the display area. For example, the reference point can be set at the position of the center of gravity of the shape of the display area.

(Coordinate system calculation unit)
The coordinate system calculation unit 24 calculates the position and orientation of a display coordinate system 88 based on the axis positions of each joint axis of the robot 1. The coordinate system calculation unit 24 calculates the position and orientation of the display coordinate system 88 on a reference coordinate system such as the reference coordinate system 87 based on the axis positions (angles in the case of rotation axes) of each joint axis of the robot 1, the relationship between the positions and orientations of the joint axes expressed by DH (Denavit Hartenberg) parameters or the like, and the position and orientation of the coordinate system of the joint axes (joint coordinate system).

Here, the relationship between the joint coordinate system and the position and orientation of the display unit (the position and orientation of the display coordinate system 88) is determined. In particular, it is preferable to select a joint coordinate system in which the position and orientation relationship with the display unit is fixed, and to calculate in advance the position and orientation relationship between the display coordinate system and the joint coordinate system. In this embodiment, it is preferable to set in advance the position and orientation relationship with the joint coordinate system set for the joint axis closest to the display unit toward the robot base 59 for the joint axes connected in sequence via links.

Regardless of the form of the robot, it is possible to calculate the position and orientation of the display coordinate system on the reference coordinate system by a method widely recognized by those skilled in the art. For example, a homogeneous transformation matrix (relationship between position and orientation) in the reference coordinate system up to the joint axis where the relationship of position and orientation with respect to the display unit is fixed is determined. From the homogeneous transformation matrix between the joint axis and the display unit, it is possible to obtain the position and orientation of the display coordinate system on the reference coordinate system.

If the robot is mounted on a device that changes the position and orientation of the robot, such as a traveling axis or another driving device, the position and orientation in the display coordinate system may be calculated taking into account the amount of movement of those driving parts. In addition, the position and orientation of the display coordinate system may be determined as the relationship between the relative position and orientation of a peripheral driving device installed separately from the robot. In this case, the relative position and orientation of the display coordinate system may be calculated taking into account changes in the position and orientation of all driving parts up to the display coordinate system, including driving devices installed separately from the robot.

(Reference direction setting section)
The reference direction setting unit 22 sets a reference direction that is a direction used as a reference when determining the orientation of an image displayed on the display unit. The reference direction setting unit 22 sets a direction on the reference coordinate system 87 of the robot 1 as the reference direction. The reference direction may also be a direction on a coordinate system that is convenient for setting the reference direction. The reference direction setting unit 22 may also set the reference direction using a coordinate system that is set so as to be fixed with respect to the space in which the robot system exists.

The reference direction setting unit 22 may set the reference direction based on a direction on the robot's reference coordinate system 87, using a reference coordinate system 87 that is set for the robot 1 and that represents the position and posture of the robot, such as each joint of the robot 1 or the arm tip. The reference direction setting unit 22 may also set the reference direction using a coordinate system set for a specified part of the robot 1. In this case, the reference direction changes with the drive or movement of the specified part of the robot.

The reference direction setting unit 22 may also set the reference direction using a reference coordinate system of a device located near the robot 1, a jig, another robot, or another driving device, or other device surrounding the robot. At this time, the reference direction setting unit 22 may set the reference direction at a portion of the surrounding device whose position and posture change due to the driving or movement of the surrounding device. In this case, the reference direction changes with the driving or movement of the surrounding device.

By having the reference direction setting unit set a reference direction in a desired coordinate system, it is possible to set a direction that makes it easier for the worker to see the image displayed on the display unit. Also, for an input image that involves an input operation, it is possible to set a direction that makes it easier to input. There are various other possible methods for setting the reference direction, and examples are shown below.

Figure 11 shows an oblique view of a reference plane to explain the first control in which the reference direction setting unit sets the reference direction. The reference direction setting unit 22 may set the reference direction in the robot's reference coordinate system 87 so as to have a predetermined attitude relationship with the normal direction at the reference point 83 of the reference plane 82 set with respect to the display area. For example, the reference direction setting unit 22 may set the direction calculated by a predetermined homogeneous transformation matrix with respect to the normal direction as the reference direction. In this example, the reference plane 82 and the display area of the display unit are coplanar. For this reason, the reference direction setting unit 22 may set the reference direction in the robot's reference coordinate system 87 so as to have a predetermined attitude relationship with the normal direction of the display area.

That is, the reference direction setting unit 22 calculates the normal direction of the reference plane 82 so as to pass through the reference point 83, as shown by arrow 82a. The reference direction setting unit 22 tilts the normal direction by the orientation and angle of arrow 91 expressed in the reference coordinate system 87. The direction shown by arrow 81 becomes a direction parallel to the reference direction. Then, the reference direction setting unit 22 sets the direction parallel to the direction shown by arrow 81 as the reference direction shown by arrow 80.

By determining the reference direction in this manner, it is possible to stably set the reference direction even if the position of each axis of the joint axes of the robot 1 changes due to the driving of the robot 1. For example, as described below, a predetermined direction in the robot's reference coordinate system, such as the direction of gravity, can be set as the reference direction. In this case, there is a problem that if the normal direction of the display area of the display unit becomes parallel to the reference direction, it becomes impossible to determine the display phase direction. However, control that sets the reference direction from the normal direction of the reference plane does not have this problem and allows the reference direction to be set stably.

Alternatively, the posture relationship between the display unit and the reference direction may change during the period when control is being performed to set a specific direction in the robot's reference coordinate system as the reference direction. In this case, the reference plane on the display area of the display unit and the reference direction may become orthogonal or close to being orthogonal, and this posture relationship may result in large changes in the display phase direction or frequent large changes in the display phase direction. As a result, calculation of the display phase direction may become unstable. In this case, it is possible to switch to control that sets the reference direction from the normal direction of the reference plane.

When the robot is driven, the image is displayed in an orientation that maintains a predetermined attitude relationship in the robot's reference coordinate system with respect to the display area of the display unit, making it possible for the worker to easily view the image displayed on the display unit. Alternatively, it is possible to display an input image that makes it easier for the worker to perform input operations.

Figure 12 shows a schematic plan view illustrating the second control in which the reference direction setting unit sets the reference direction. The reference direction setting unit 22 may set the reference direction according to the range of positions where the display unit exists and the range of attitudes of the display unit on a reference coordinate system such as the robot's reference coordinate system 87. The position where the display unit exists may be the position where the origin or reference point of the display coordinate system exists. The attitude of the display coordinate system may be used as the attitude of the display unit. For example, when the position of the display unit changes as the robot is driven, the reference direction may be changed depending on the position where the display unit exists on the reference coordinate system.

The range for setting the reference direction according to the position where the display unit is located may be a range defined by two coordinate axes of the robot's reference coordinate system. The range for determining the position where the display unit is located may be a range defined by one or three coordinate axes of the robot's reference coordinate system.

In the example shown in FIG. 12, four ranges 101a, 101b, 101c, and 101d are defined based on two coordinate axes (X-axis and Y-axis) of the robot's reference coordinate system 87. The reference direction setting unit 22 can set the reference direction indicated by the arrow 80a when the display unit is located within the range 101a. The reference direction setting unit 22 can set the reference direction indicated by the arrow 80b when the display unit is located within the range 101b. Similarly to this control, the reference direction setting unit 22 can set the reference directions indicated by the arrows 80c and 80d when the display unit is located within the ranges 101c and 101d. The reference directions in the respective ranges 101a, 101b, 101c, and 101d can be set in the reference coordinate system 87.

By setting the reference direction in this way, the image displayed by the display unit can be made easier for the worker to see depending on the range of positions or postures in which the display unit is located when the robot is operating. It also makes it easier for the worker to input information into the input image. For example, when multiple workers are working around the robot, changing the reference direction depending on the worker's working range makes it easier for each worker to see the display.

Figure 13 shows a first schematic diagram of the robot illustrating the third control in which the reference direction setting unit sets the reference direction. Figure 14 shows a second schematic diagram of the robot illustrating the third control in which the reference direction setting unit sets the reference direction. Figure 15 shows a third schematic diagram of the robot illustrating the third control in which the reference direction setting unit sets the reference direction. In addition to the schematic diagram of the robot, Figures 13 to 15 show an enlarged view of image 71a displayed on display unit 61a.

In Figure 13, the robot 1 is placed above the installation surface 105. In Figure 14, the robot 1 is placed below the installation surface 105. In Figure 14, the robot 1 is suspended. In Figure 15, the installation surface 105 is inclined. In both robots 1, the base part 14 is fixed to the installation surface 105. The arrow 90 indicates the direction of gravity (vertically downward).

The reference direction setting unit 22 may set the reference direction to the direction of gravity, the opposite direction to the direction of gravity, or a direction obtained by rotating the direction of gravity in a predetermined posture. That is, the reference direction setting unit 22 may set the direction determined by the reference coordinate system 87 as the reference direction. In the examples shown in Figures 13 to 15, the reference direction is set to the vertical upward direction (opposite the direction indicated by the arrow 90). Each display unit 61a is arranged in various orientations. However, in any orientation of the display unit 61a, the display control unit 27 displays the image 71a so that the vertical upward direction is the top of the image.

The third control is effective when the normal direction of the display area (reference plane) of the display unit forms an angle of a certain degree or more with the direction of gravity. The reference direction determined by this control is based on the direction of gravity, and is therefore intuitively easy for the worker to understand. For example, the display device may be controlled to display an image at a predetermined angle of 0 degrees or more with respect to the horizontal direction, regardless of the angle of the joint axis of the robot on which the display unit is provided. Also, when the display unit is configured as a touch panel for teaching operations, input images such as displayed buttons may also be controlled to be displayed at a predetermined angle with respect to the horizontal direction.

By setting the reference direction based on the direction of gravity in this way, the image displayed on the display unit becomes easier for the worker to see, regardless of the installation state of the robot or the posture of the robot. It also makes it easier for the worker to input information into the input image.

Figure 16 shows an oblique view of a reference plane to explain the fourth control in which the reference direction setting unit sets the reference direction. The angle between the reference direction and the normal direction of the reference plane may change due to the driving of the robot, and the angle between the reference direction and the normal direction of the reference plane may become smaller than a predetermined threshold. For example, a change in the position and posture of the robot 1 may cause the angle θ1 between the reference direction indicated by arrow 80 and the normal direction of the reference plane 82 indicated by arrow 82a to become smaller than a predetermined threshold.

In this case, the reference direction setting unit 22 may not change the reference direction before the angle θ1 becomes smaller than the predetermined threshold. In other words, when the position and posture of the robot change and the angle θ1 becomes smaller than the predetermined threshold, the reference direction setting unit 22 may maintain the previous reference direction.

When the relationship between the display area of the display unit and the reference direction changes due to the robot's operation, and the reference direction becomes closer to the normal direction of the reference plane 82, it may become impossible to stably calculate the orientation of the image on the display unit (display phase direction). In such a situation, the image orientation can be determined and the image can be displayed by continuing the reference direction without changing it.

Figure 17 shows a perspective view of a reference plane to explain the fifth control in which the reference direction setting unit sets the reference direction. The reference direction setting unit 22 may set the reference direction based on a position specified by the operator on the display area of the display unit. In this example, the reference plane 82 is set to be on the same plane as the display area of the display unit.

The worker changes the position and posture of the robot 1 so that the reference plane 82 is in the desired posture. The display control unit 27 displays a reference point 83 in the display area of the display unit. If the display unit is configured as a touch panel type display panel, the worker designates a specified point 102 by pressing a desired point in the display area of the display unit. The reference direction setting unit 22 calculates the direction from the reference point 83 indicated by the arrow 81 to the specified point 102. The reference direction setting unit 22 sets a direction parallel to the arrow 81 as the reference direction indicated by the arrow 80. The reference direction can be set in the reference coordinate system of the robot.

In the fifth control, the reference direction is set as a direction within the plane of the display area of the display unit. Here, if the desired reference direction is a direction that intersects with the plane of the display area of the display unit, the worker can change the position and posture of the robot to change the orientation of the display area of the display unit.

In the fifth control, the reference direction can be easily set so that the image is oriented in a direction that is convenient for the worker depending on the state of the robot. This control makes it easy to set a reference direction that makes it easier for the worker to see the display on the display unit. It also makes it easier for the worker to input information into the input image.

Depending on the shape of the display area, the display area and the reference plane may not be on the same plane. For example, the display area of the display unit may be formed in a curved shape. In this case, the reference direction setting unit can project a specified point specified by the operator onto the reference plane in a predetermined direction. Then, the reference direction may be set based on the point projected onto the reference plane.

In the above embodiment, the direction from the reference point toward the specified point is set as the reference direction, but this is not limited to the above embodiment. Any direction based on a specified point specified in the display area by the operator can be set as the reference direction. For example, in the first control for setting the reference direction, when setting the orientation of the reference direction relative to the normal direction of the reference plane, control may be performed to specify a specified point on the display area.

(Display phase calculation section)
FIG. 18 shows a perspective view of a reference plane to explain the first control for calculating the display phase direction. The display phase calculation unit 25 calculates a display phase direction (direction indicated by an arrow 84) which is a reference direction for the display orientation on the display area of the display unit by projecting a reference direction indicated by an arrow 80 onto a reference plane 82 which is a reference plane of the display area of the display unit. FIG. 18 shows a case where the reference direction indicated by the arrow 80 and the normal direction of the reference plane 82 indicated by an arrow 82a are orthogonal. In this case, the display phase calculation unit 25 can project the reference direction in an arbitrary direction as indicated by an arrow 92. The display phase calculation unit 25 projects the reference direction toward the reference plane 82. Then, the display phase direction indicated by an arrow 84 can be determined so as to pass through the reference point 83.

Figure 19 shows an oblique view of a reference plane to explain the second control for calculating the display phase direction. Figure 19 shows a case where the reference direction indicated by arrow 80 is inclined and not perpendicular to the normal direction of reference plane 82 indicated by arrow 82a. Based on the reference direction indicated by arrow 80, the display phase calculation unit 25 calculates a direction parallel to the reference direction passing through reference point 83 as indicated by arrow 81.

The display phase calculation unit 25 projects a direction parallel to the reference direction passing through the reference point 83 onto the normal direction of the display area, as shown by arrow 92. The display phase calculation unit 25 projects a direction parallel to the reference direction passing through the reference point 83 onto the reference plane 82. This control makes it possible to determine a display phase direction that serves as the reference for the phase of the display on the display coordinate system, as shown by arrow 84.

Alternatively, the display phase calculation unit 25 calculates the direction in which the reference direction indicated by the arrow 80 is projected onto the reference plane 82 toward the normal direction of the display area. Then, the display phase calculation unit 25 calculates a direction passing through the reference point 83 so as to be parallel to the projected direction. The display phase calculation unit 25 can determine this direction as the display phase direction.

Figure 20 shows a schematic diagram of a robot illustrating the third control for calculating the display phase direction. Figure 21 shows a plan view of a reference plane illustrating the third control for calculating the display phase direction. With reference to Figures 3, 20, and 21, the display phase calculation unit 25 projects a reference direction onto a reference plane 82, which is a reference plane for the display area of the display unit 61a. Furthermore, the display phase calculation unit 25 may calculate the display phase direction by rotating at a predetermined offset angle around a reference point 83, which is a point that serves as a reference for display on the display area of the display unit.

Here, the reference direction is defined as a vertically upward direction, as indicated by arrow 80. The display phase calculation unit 25 calculates a direction parallel to the reference direction on the reference plane 82 by projecting the reference direction onto the reference plane 82. The display phase calculation unit 25 calculates a direction parallel to the reference direction passing through the reference point 83, as indicated by arrow 81.

The processing unit 21 of this embodiment includes an offset angle setting unit 26 that sets an offset angle that is a predetermined angle centered on a reference point on the display area. The offset angle setting unit 26 sets the offset angle in response to an operator's operation. For example, the operator inputs the offset angle θ2 using a teaching operation panel. Then, the offset angle setting unit 26 can set the offset angle θ2 relative to the direction indicated by the arrow 81 that projects the reference direction onto the reference plane.

Next, the display phase calculation unit 25 rotates the direction indicated by the arrow 81 at an offset angle θ2, which is a predetermined rotation angle, around the reference point 83. The display phase calculation unit 25 rotates the direction in which the reference direction is projected within the reference plane 82. Then, the display phase calculation unit 25 calculates the display phase direction indicated by the arrow 84. The display control unit 27 controls and displays the orientation of the image on the display area of the display unit 61a based on the display phase direction calculated by the display phase calculation unit 25 and the reference point 83.

As shown in FIG. 20, image 71a displayed on display unit 61a is tilted at an offset angle θ2 with respect to the direction in which the reference direction is projected onto the reference plane. This control makes it possible to easily offset the orientation of the image displayed in the display area of the display unit for the worker. The image can be displayed so that it has a specified phase relationship with a specified direction based on the reference direction. When the worker views the display unit from a specific direction, information is displayed according to the specific direction, making the information easier to see. Furthermore, when the display device has an input function, it becomes easier for the worker to perform input operations on the input image.

Figure 22 shows a plan view of the reference plane to explain the fourth control for setting the display phase direction. In this example, the reference plane 82 and the display area of the display unit are arranged on the same plane. The offset angle setting unit 26 is formed so that the offset angle θ2 can be set by the operator operating the display area of the display unit.

The display control unit 27 displays the direction in which the current reference direction is projected onto the reference plane 82, as shown by arrow 81. The operator presses the display area of the display unit to specify a specified point 103. The offset angle setting unit 26 calculates an offset angle θ2 with respect to the direction shown by arrow 81. The display phase calculation unit 25 sets a display phase direction based on the offset angle θ2. The display control unit 27 controls and displays the orientation of the image on the display area of the display unit 61a based on the display phase direction calculated by the display phase calculation unit 25 and the reference point 83.

If the display area of the display unit is curved, the offset angle may be set based on the point where the operator touches the display area and projects it onto a reference plane. For example, the offset angle may be calculated based on the point where the operator projects a specified point in the normal direction of the reference plane. Alternatively, the control of rotating the image based on such an offset angle may not be performed.

(Display control unit)
The display control unit 27 controls the image displayed in the display area of the display unit. When the display unit displays an input image, the display control unit 27 controls the orientation of the input image for display. The display control unit 27 controls the orientation of the image on the display area of the display unit so as to display an image that has a predetermined posture relationship with respect to the reference direction, based on the axis positions of each axis of the joint axes of the robot and the reference direction.

In addition, the display control unit 27 controls and displays the position and orientation of the image on the display area of the display unit based on the axis positions of each axis of the robot's joint axes, the reference direction, the reference point on the display coordinate system, and the display phase direction on the display coordinate system calculated by the display phase calculation unit 25.

In addition, the robot may be mounted on a device that changes the position and posture of the robot, such as a device having a traveling axis or another driving device. In this case, it is preferable that the processing unit 21 displays an image based on the amount of movement of these driving parts along with the axis positions of each axis of the robot's joint axes.

By controlling the orientation of the image on the display area of the display unit by the display control unit 27, when the attitude of the display unit changes due to the driving of the robot, the image on the display area of the display unit can be displayed in a desired orientation. Alternatively, an input image can be displayed in a desired orientation to perform an input operation.

Next, a specific example of an image displayed by the display control unit 27 will be described. Figure 23 shows a schematic side view of the robot when the robot is in the first position and posture. Here, a case will be described where the reference direction indicated by the arrow 80 for the image on the display unit 61a is oriented vertically upward. Such a reference direction can be set, for example, by the first control or the third control for setting the reference direction in this embodiment. In the display unit 61a, the display phase direction indicated by the arrow 84 faces in the same direction as the reference direction indicated by the arrow 80. In other words, the display phase direction indicated by the arrow 84 faces vertically upward.

Figure 24 shows a schematic diagram of the robot when driven from the first position and posture. Figure 25 shows a schematic diagram of a comparative example robot when driven from the first position and posture. Here, the axis position of the J2 axis 52 between the swivel base 13 and the lower arm 12 has changed. As indicated by arrow 93, the lower arm 12 has rotated and its posture has changed.

Figure 24 shows the state of the image on the display unit 61a when display control is performed by the processing unit 21 of this embodiment. The display control unit 27 controls the orientation of the image on the display area of the display unit 61a so as to display an image that has a predetermined posture relationship with respect to the reference direction, based on the axis positions of each axis of the joint axes of the robot 1 and the reference direction. The display phase direction indicated by the arrow 84 remains parallel to the reference direction. In other words, the display phase direction is maintained facing vertically upward.

In the comparative example shown in FIG. 25, the display control of this embodiment is not implemented. In the display unit 61a, the display phase direction indicated by the arrow 85 rotates together with the rotation of the lower arm 12. As a result, the image displayed on the display unit 61a becomes tilted. The worker has difficulty viewing the image displayed on the display unit 61a.

In contrast, referring to Fig. 24, by implementing display control by the processing unit 21 of this embodiment, even if the axis position of the joint axis changes, the display phase direction indicated by the arrow 84 faces upward in the vertical direction. This maintains an easy-to-view state of the image displayed on the display unit 61a.

Figure 26 shows a schematic side view of the robot when it is in the second position and posture. Figure 27 shows a schematic plan view of the robot when it is in the second position and posture. In this example, the display area (reference plane) of display unit 63b is parallel to the horizontal direction. For display unit 63b, the reference direction indicated by arrow 80 is set to the horizontal direction. Such a reference direction can be set by the first control, second control, or third control that sets the reference direction in this embodiment.

On the display unit 63b, the display phase direction indicated by the arrow 84 is oriented horizontally. Also, the worker views the display unit 63b from above the robot 1, as indicated by the arrow 94. The display phase direction indicated by the arrow 84 is oriented in a direction that allows the worker to easily view the image.

Figure 28 shows a schematic diagram of the robot when driven from the second position and posture. Figure 29 shows a schematic diagram of a comparative example robot when driven from the second position and posture. In this example, the swivel base 13 rotates around the J1 axis 51 as shown by the arrow 95.

In the example shown in Fig. 28, the display is controlled by the processing unit 21. The display control unit 27 controls the orientation of the image on the display area of the display unit based on the axis position and reference direction of each axis of the robot's joint axes, so as to display the image in a predetermined posture relationship with the reference direction. The display phase direction indicated by arrow 84 is parallel to the reference direction indicated by arrow 80.

In the comparative example shown in Figure 29, the display control of this embodiment is not implemented. The display phase direction indicated by the arrow 85 on the display unit 63b changes orientation with the rotation of the upper arm 11. As a result, when the worker looks at the display unit 63b, the image displayed on the display unit 63b is tilted. The worker has difficulty viewing the image displayed on the display unit 63b.

With reference to Figure 28, by implementing display control by the processing unit 21 in this embodiment, even if the axis position of the joint axis changes, the display phase direction indicated by the arrow 84 remains parallel to the reference direction. The image displayed on the display unit 63b is displayed in the same orientation even when the robot is driven. This makes it possible to maintain a state in which the image displayed on the display unit 63b is easy for the worker to view.

Figure 30 shows a schematic side view of the robot when it is in the third position and posture. The upper arm 11 to which the display unit 63b is fixed is tilted with respect to the horizontal direction. For the display unit 63b, the reference direction indicated by the arrow 80 is set to face the horizontal direction. Such a reference direction can be set by the second control or the third control for setting the reference direction in this embodiment. In this example, the normal direction of the display area (reference plane) of the display unit 63b and the reference direction are not perpendicular to each other. The worker then looks at the display unit 63b in the direction indicated by the arrow 96.

Figure 31 shows an explanatory diagram of the display phase direction when the robot is driven from the third position and posture. With reference to Figures 30 and 31, in this example, the swivel base 13 rotates around the J1 axis 51 as indicated by arrow 95. In the example shown in Figure 31, display control is performed by the processing unit 21. The display control unit 27 controls and displays the orientation of the image on the display area of the display unit so as to display an image that has a predetermined posture relationship with respect to the reference direction based on the axis positions of each axis of the robot's joint axes and the reference direction.

Even when the upper arm 11 rotates as shown by arrows 95a and 95b, the display phase direction shown by arrow 84 is maintained in the direction in which the reference direction is projected onto the display area (reference plane) of the display unit 63b. The display phase direction shown by arrow 84 is the direction that makes it easy for the operator to view the image. When the operator looks at the display unit 63b in the direction shown by arrow 96, the image is displayed in the same orientation.

Figure 32 shows an explanatory diagram of the display phase direction in a comparative example when the robot is driven from the third position and posture. In the comparative example, the display control of this embodiment is not implemented. The display phase direction indicated by arrow 85 on display unit 63b changes orientation with the rotation of upper arm 11 indicated by arrows 95a and 95b. As a result, when a worker looks at display unit 63b, the image displayed on display unit 63b is tilted. The worker has difficulty seeing the image displayed on display unit 63b.

With reference to Fig. 31, by implementing display control by the processing unit 21 of this embodiment, even if the axis position of the joint axis changes, the display phase direction is maintained in a direction that allows the operator to easily view the image. This makes it possible to maintain a state in which the image displayed on the display unit 63b is easily viewable by the operator.

In the above-described embodiment, a display coordinate system, a reference plane, and a reference point are set for the display area of the display unit to calculate the display phase direction, but this is not limited to the embodiment. The processing unit does not have to set a display coordinate system, a reference plane, and a reference point. The processing unit can calculate the display phase direction by any control based on the reference direction. For example, when the reference direction and the display area of the display unit are parallel, the processing unit may set the reference direction to the display phase direction.

Furthermore, the display control unit may perform image processing on the image generated on the reference plane. For example, any image conversion operation can be performed, such as enlarging an image, reducing an image, or performing a predetermined image conversion. The display control unit can then display the converted image on the display unit. For example, as described above, when the display area of the display unit is curved, an image on the reference plane can be projected onto the display area of the display unit to generate an image to be displayed in the display area.

(Example of image displayed on the display)
Next, an example of an image displayed in the display area of the display unit by the display control unit will be described. Here, an example of an image displayed on the display unit 61a will be described.

33 shows a fourth image displayed on the display unit. FIG. 34 shows a fifth image displayed on the display unit. The fourth image 71c is an image showing the driving state of the robot. The fifth image 72b is an input image for inputting the driving speed around the joint axis of the joint axis on which the display unit is arranged or a desired joint axis. In addition, there are cases where joint axes other than one joint axis are driven together with one joint axis. In this case, when changing the driving speed around one joint axis, the driving speeds of all joint axes may be adjusted so that the position and posture of the tip of the robot do not change. In addition, the moving speed of the tip of the robot in the operating program being executed may be changed in the input image displayed on the display unit. The display control unit 27 can display information or an input image in the display area of the display unit in at least one of characters, figures, symbols, colors, and patterns.

Furthermore, the display control unit 27 may display at least one of a character, a figure, a symbol, a color, and a pattern to indicate the orientation of the image. In the fourth image 71c and the fifth image 72b, an arrow image 73 indicating the upper side of the image is displayed.

In this way, the display control unit generates an image in the display area so that the orientation of the image can be seen, making it easier for the worker to recognize the orientation of the image in the display area on the display unit. In addition, when the display unit displays an input image, the operability for the worker can be improved.

Figure 35 shows the sixth image displayed on the display unit. Figure 36 shows the seventh image displayed on the display unit. In the sixth image 71d, the character "upper" is displayed to indicate the upper side of the image. In the seventh image 71e, part of the frame surrounding the information indicating the driving status is recessed to indicate the upper side of the image.

Figure 37 shows the eighth image displayed on the display unit. Figure 38 shows the ninth image displayed on the display unit. In the eighth image 71f and the ninth image 71g, a pattern indicating the upper side is displayed at the top. Furthermore, in the ninth image 71g, the pattern indicating the upper side is displayed in a color different from the background color of the display area. In this way, the display control unit 27 can generate images that indicate the orientation of the image in various forms.

In the above embodiment, an image showing the top side has been described, but the present invention is not limited to this embodiment. The display control unit may display the image so that the orientation of the image can be understood. For example, the display control unit may display text or the like showing the bottom side of the image.

Figure 39 shows a tenth image displayed on the display unit. Figure 40 shows an eleventh image displayed on the display unit. The display control unit 27 may implement control to change the size of an image of a predetermined portion on the display area of the display unit 61a depending on the movement speed of the display unit 61a of the display device accompanying the movement of the robot. For example, the display control unit 27 may display the predetermined image small when the movement speed of the display unit 61a is slow, and display the predetermined image large when the movement speed of the display unit 61a is fast.

In the tenth images 71ha and 71hb of FIG. 39, the number of times the task has been performed is displayed on the display unit 61a. The display control unit 27 can change the size of the image of the number of times the task has been performed, as shown by arrow 97, depending on the movement speed of the display unit 61a. For example, the display control unit 27 can calculate the movement speed of the display coordinate system as the movement speed of the display unit 61a. The display control unit 27 acquires the position and time of the origin of the display coordinate system at predetermined time intervals. The display control unit 27 can calculate the movement speed of the origin of the display coordinate system based on the position and time of the origin of the display coordinate system.

When the movement speed of the display unit 61a is lower than a predetermined threshold, the display control unit 27 can display the image of the number of times performed small as shown in image 71ha. When the movement speed of the display unit 61a is equal to or higher than a predetermined threshold, the display control unit 27 can display the image of the number of times performed large as shown in image 71hb. By implementing this control, the image of the desired part is displayed large when the robot's driving speed is high, making it easier for the worker to see the displayed image.

The eleventh images 72ca and 72cb in Fig. 40 are input images for the worker to perform input operations. In the eleventh images 72ca and 72cb, a predetermined setting value can be changed by pressing the "+" button or the "-" button. The display control unit 27 can change the size of the image showing the setting value, as shown by the arrow 98, depending on the moving speed of the display unit 61a. The display control unit 27 can also change the size of the button.

When the moving speed of the display unit 61a is lower than a predetermined threshold, the display control unit 27 can display the image of the setting value large and the image of the button small, as shown in image 72ca. When the moving speed of the display unit 61a is equal to or higher than a predetermined threshold, the display control unit 27 can display the image of the setting value small and the image of the button large, as shown in image 72cb . With this control, even if the driving speed of the robot increases, the button image becomes large, allowing the worker to stably press the button.

In the robot system of this embodiment, when the display device has an input function, the worker can perform input operations while the robot is moving. By implementing control to change the size of the image according to the moving speed of the display unit, the worker can easily see the image and perform input operations. For example, by displaying the image of the button in a larger size when the moving speed of the display unit increases, the worker can easily perform input operations. In addition, when allowing input operations from the display device while the robot is moving, it is preferable to set a limit on the moving speed of the robot or the drive speed of the joint axis at which input operations can be performed for safety. For example, it is preferable to allow input operations when the moving speed of the robot or the drive speed of the joint axis is equal to or lower than a predetermined speed. In addition, the setting value at which input operations can be performed may be changed depending on the moving speed of the robot or the drive speed of the joint axis.

In each of the above-mentioned controls, the order of steps can be changed as appropriate as long as the functions and actions are not changed. The above-mentioned embodiments can be combined as appropriate. In each of the above-mentioned figures, the same or equivalent parts are given the same reference numerals. Note that the above-mentioned embodiments are examples and do not limit the invention. Furthermore, the embodiments include modifications of the embodiments shown in the claims.

REFERENCE SIGNS LIST 1 Robot 2 Hand 4 Control device 6 Robot system 11 Upper arm 12 Lower arm 13 Swivel base 14 Base section 15 Wrist 16 Flange 21 Processing section 22 Reference direction setting section 23 Coordinate system setting section 24 Coordinate system calculation section 25 Display phase calculation section 26 Offset angle setting section 27 Display control section 42 Memory section 43 Operation command section 51 J1 axis 52 J2 axis 53 J3 axis 54 J4 axis 55 J5 axis 56 J6 axis 59 Base R1, R2 Rotation axis 60, 61a, 61b, 61c, 62, 63a, 63b, 64, 65, 66, 67 Display section 71a, 71b, 71c, 71d, 71e, 71f, 71g, 71ha, 71hb Images 72a, 72b, 72ca, 72cb Images 82 Reference plane 83 Reference point 87 Reference coordinate system 88 Display coordinate system 101a to 101d Range 102, 103 Designated point θ1 Angle θ2 Offset angle

Claims (15)

A robot including a plurality of joint axes;
A display unit disposed on a main body portion of the robot;
a display control unit that controls an image to be displayed in a display area of the display unit;
a reference direction setting unit that sets a reference direction that is a direction used as a reference when determining the orientation of an image to be displayed on the display unit;
a coordinate system setting unit that sets a display coordinate system which is a coordinate system set for a display area of the display unit, a reference plane which is a reference plane for the display area of the display unit and is set on the display coordinate system, and a reference point which is a point serving as a reference for display on the display area of the display unit and is set on the display coordinate system;
a coordinate system calculation unit that calculates a position and a posture of the display coordinate system based on the axis positions of each joint axis of the robot;
a display phase calculation unit that calculates a display phase direction that is a reference direction for a display orientation on a display area of the display unit by projecting a reference direction onto a reference plane on a display coordinate system ,
The robot system is characterized in that the display control unit controls and displays an image orientation on the display area of the display unit based on the display phase direction calculated by the display phase calculation unit and the reference point. A robot including a plurality of joint axes;
A display unit disposed on a main body portion of the robot;
a display control unit that controls an image to be displayed in a display area of the display unit;
a reference direction setting unit that sets a reference direction that is a direction used as a reference when determining the orientation of an image to be displayed on the display unit;
a coordinate system setting unit that sets a display coordinate system which is a coordinate system set for a display area of the display unit, a reference plane which is a reference plane for the display area of the display unit and is set on the display coordinate system, and a reference point which is a point serving as a reference for display on the display area of the display unit and is set on the display coordinate system;
a coordinate system calculation unit that calculates a position and a posture of the display coordinate system based on the axis positions of each joint axis of the robot;
a display phase calculation unit that projects a reference direction onto a reference plane on a display coordinate system, and further rotates the reference direction by an offset angle that is a predetermined angle around the reference point, thereby calculating a display phase direction that is a direction that is a reference for the orientation of display on a display area of the display unit,
The robot system is characterized in that the display control unit controls and displays an image orientation on the display area of the display unit based on the display phase direction calculated by the display phase calculation unit and the reference point . the display unit has an input function of displaying an input image for inputting information and inputting information by an operation corresponding to the input image;
The robot system according to claim 1 , wherein the display control unit controls an orientation of the input image to display the input image. 3. The robot system according to claim 1 , wherein the reference direction setting unit sets the reference direction so as to have a predetermined posture relationship with a normal direction of a reference plane on a reference coordinate system of the robot. The robot system according to claim 1 , wherein the reference direction setting unit sets a reference direction in a reference coordinate system of the robot according to a range in which the display unit exists. The robot system according to claim 1 , wherein the reference direction setting unit sets, as the reference direction, a direction of gravity, a direction opposite to the direction of gravity, or a direction obtained by rotating the direction of gravity in a predetermined posture. The robot system according to claim 1 , wherein the reference direction setting unit sets the reference direction based on a position specified by an operator on a display area of the display unit. 3. The robot system according to claim 1, wherein, when an angle between the reference direction and a normal direction to a reference plane changes due to driving of the robot and the angle with the normal direction becomes smaller than a predetermined threshold, the reference direction setting unit maintains, without changing, the reference direction before the angle with the normal direction became smaller than the predetermined threshold. 3. The robot system according to claim 2 , further comprising an offset angle setting unit that sets an offset angle that is a predetermined angle centered on the reference point, based on a position specified by an operator on a display area of the display unit. 4. The robot system of claim 3, wherein when information is input on the display unit, the display unit displays an image that accepts input of specific information to enable an operator to perform an input operation, and when the operator inputs the specific information, the image is switched to one that allows the operator to perform an input operation. The robot system according to claim 3 , wherein the display unit is configured so that at least one of an authority to perform an input operation and an authority to display information can be changed by a predetermined input operation by an operator. The robot system according to claim 1 , wherein the display control unit displays at least one of a character, a figure, a symbol, a color, and a pattern on the display area of the display unit to indicate the orientation of the image. The robot system according to claim 1 , wherein the display control unit controls the size of an image of a predetermined portion on a display area of the display unit to change depending on a moving speed of the display unit accompanying the driving of the robot. The robot system according to claim 1 , wherein the display unit is detachable from a main body of the robot. A robot including a plurality of joint axes;
A display unit disposed on a main body portion of the robot;
a display control unit that controls an image to be displayed in a display area of the display unit;
a reference direction setting unit that sets a reference direction that is a direction used as a reference when determining the orientation of an image to be displayed on the display unit,
the display control unit controls and displays an image in a display area of the display unit so as to display the image in a predetermined posture relationship with respect to a reference direction, based on an axis position of each joint axis of the robot and a reference direction;
The robot system according to claim 1, wherein the reference direction setting unit sets a reference direction based on a position specified by an operator on a display area of the display unit.

Images