JP7838643B2 - Operation support system, operation support device, operation support method, and operation support program - Google Patents

Description

This invention relates to an operation support system, an operation support device, an operation support method, and an operation support program.

Traditionally, applications have been provided with a GUI (Graphical User Interface) tailored to the user's actions in order to restrict or assist their actions. However, there are many users of applications, and each person's operating methods vary. Furthermore, these operating methods may change over time due to changes in preferences, etc. Therefore, it is preferable to change the GUI of the application to accommodate these differences between users and changes over time.

However, modifying the application itself can sometimes require significant costs and a vast amount of time. In particular, if the application is a business application, the costs and time required for modification are likely to be enormous. Therefore, a UI extension technology has been proposed that improves operator productivity by overlaying functional enhancements such as text input boxes and input validation onto the web screen, without modifying the application itself.

Japanese Patent Publication No. 2017-072872

However, from the perspective of protecting personal information and confidential company information, the GUI displayed on the screen may contain information that should be kept secret, and depending on the environment in which the terminal is operated, it may be undesirable to display such confidential information on the monitor. For example, when conducting business negotiations with a customer while showing a screen with a GUI, it is generally advisable to refrain from displaying the minimum customer contract amount on the monitor, which is not typically disclosed to customers. In such cases, extra work may be required, such as investigating the confidential information in an environment inaccessible to the customer, or using other means to confirm the content of the confidential information to be concealed, which may lead to a decrease in the user's work efficiency.

The present invention has been made in view of the above, and aims to improve the work efficiency of tasks using applications having a GUI.

To solve the above-mentioned problems and achieve the objective, the operation support system comprises a first information processing device and a second information processing device. The first information processing device includes a display control unit that masks predetermined information contained in an image with a first UI component and displays the masked image on a screen together with a second UI component related to the predetermined information. The second information processing device includes a reading unit that reads the second UI component on the image displayed on the screen of the first information processing device, an extraction unit that extracts information related to the predetermined information from the second UI component read by the reading unit, a voice playback control unit that plays back the predetermined information and inquiry information prompting input related to the predetermined information based on the information related to the predetermined information extracted by the extraction unit , and an operation information reflection unit that receives input of operation instructions for the inquiry information, notifies the display control unit of the operation instructions, and causes the display control unit to reflect the operation instructions on the image displayed on the screen .

According to the present invention, it is possible to improve the work efficiency of tasks performed using applications with a GUI.

Figure 1 is a block diagram of the operation support system according to the first embodiment. Figure 2 is a diagram illustrating the general operation of the operation support system. Figure 3 is a flowchart of the image display processing by the work terminal according to the first embodiment. Figure 4 is a flowchart of the process for acquiring confidential information by an external terminal according to the first embodiment. Figure 5 is a block diagram of the operation support system according to the second embodiment. Figure 6 is a flowchart of the process for acquiring confidential information by an external terminal according to the second embodiment. Figure 7 is a block diagram of the operation support system according to the third embodiment. Figure 8 is a flowchart of the process for acquiring confidential information by an external terminal according to the third embodiment. Figure 9 is a flowchart of the image display processing by the work terminal according to the third embodiment. Figure 10 shows an example of a computer that runs a confidential information acquisition program and an image display program included in the operation support program.

Below, an embodiment of the operation support system, operation support device, operation support method, and operation support program disclosed in this application will be described in detail with reference to the drawings. However, the operation support system, operation support device, operation support method, and operation support program disclosed in this application are not limited to the embodiment described below.

[First Embodiment]
[Configuration of the operation support system]
Figure 1 is a block diagram of the operation support system according to the first embodiment. The configuration of the operation support system will be explained using Figure 1. The operation support system 1 includes an external terminal 10, a work terminal 20, and an audio output device 30. The operation support system 1 may also include a data storage device 40. The operation support system 1 displays an image 210 that provides various information on the work terminal 20 used by the user. If there is confidential information that is not desirable to display on the screen, the system masks the confidential information to hide it and displays a UI component with the confidential information encrypted. The operation support system 1 then has the external terminal 10 used by the user read the image 210, decrypt the UI component with the encrypted confidential information to obtain the confidential information, and output the confidential information as audio data to the audio output device 30. The details of the external terminal 10 and the work terminal 20 will be explained below.

[Work terminal]
The work terminal 20 is a computer used to perform tasks such as presenting screens to customers and inputting data on the spot when conducting business negotiations with customers. The work terminal 20 is, for example, a laptop computer or a tablet device. The work terminal 20 is an example of the "first information processing device". The work terminal 20 includes a display device 201, a display control unit 202, a linkage control unit 203, and a database 204.

The display device 201 is an LCD display or a monitor. For example, the display device 201 displays an image 210 of a presentation document used in a business negotiation.

The database 204 is a storage device for storing data. For example, the database 204 stores masking rules 51 and decoding information 52. The database 204 may also store various UI components, including masking components.

The masking rules 51 register rules for performing masking, such as information on what kind of confidential information to mask and how to mask it, as well as encryption methods. For example, there is a masking rule 51 for each type of confidential information. If the masking component 211, which is a UI component that performs masking, contains an ID marker 212, the masking rule 51 includes a method for registering encrypted data of the confidential information to the ID marker 212 contained in the masking component 211. If the masking component 211 does not contain an ID marker 212, the masking rule 51 includes the ID marker 212 to be used in the masking and a method for registering encrypted confidential information to that ID marker 212. The masking rules 51 are pre-registered in the database 204.

Furthermore, the decryption information 52 is information for decrypting the encrypted confidential information contained in each ID marker 212 registered in the masking rule 51. Decryption information 52 exists for each ID marker 212, and the decryption information corresponding to each ID marker 212 is registered in association with the ID read from the ID marker 212. The decryption information 52 is pre-registered in the database 204. However, the decryption information 52 may also be generated when the display control unit 202 registers the encrypted confidential information to the ID marker 212.

The display control unit 202 causes the display device 201 to display the image 210. For example, the display control unit 202 receives input from the user, generates an image 210 of the specified material, and displays the generated image 210 on the display device 201.

Furthermore, when generating an image 210, the display control unit 202 refers to the masking rules 51 stored in the database 204 to determine whether or not the image 210 is subject to masking. For example, the display control unit 202 determines that the image 210 is subject to masking if the characters contained in the image 210 include characters that are subject to masking according to the masking rules 51.

If image 210 is to be masked, the display control unit 202 identifies the masking area according to the masking rule 51. Furthermore, the display control unit 202 acquires the masking component 211 according to the masking rule 51. If the acquisition of the masking component 211 fails, the display control unit 202 notifies the user of the error, such as by displaying an error on the display device 201. On the other hand, if the acquisition of the masking component 211 is successful, the display control unit 202 determines whether or not the masking component 211 contains an ID marker 212.

If an ID marker 212 is present, the display control unit 202 registers encrypted confidential information to the ID marker 212 contained in the masking component 211. At this time, the display control unit 202 assigns an ID, which is identification information, to the ID marker 212 contained in the masking component 211. Subsequently, the display control unit 202 displays the image 210 masked by the masking component 211 on the display device 201.

If the ID marker 212 is not present, the display control unit 202 acquires the ID marker 212 corresponding to the masking component 211. The display control unit 202 then registers encrypted confidential information to the acquired ID marker 212. At this time, the display control unit 202 assigns an ID, which is identification information, to the ID marker 212. Subsequently, the display control unit 202 displays the image 210, which is masked by the masking component 211 and has the ID marker 212 added, on the display device 201. If the acquisition of the ID marker 212 fails, the display control unit 202 displays the image 210, which is masked by the masking component 211, on the display device 201 without adding the ID marker 212.

If the decryption information 52 is not already stored in the database 204, the display control unit 202 may generate the decryption information 52 for decrypting the encrypted confidential information when it registers the encrypted confidential information in the ID marker 212 and store it in the database 204.

Here, the masking component 211 is an example of a "first UI component," and the ID marker 212 is an example of a "second UI component." That is, the display control unit 202 masks predetermined information contained in the image 210 with the first UI component and displays the masked image 210 on the screen together with the second UI component related to the predetermined information. The display control unit 202 may also add identification information to the second UI component. Alternatively, the display control unit 202 may mask the image 210 using the first UI component which includes the second UI component.

The linkage control unit 203 communicates with the linkage control unit 107 of the external terminal 10 by sending and receiving data, thereby coordinating between the work terminal 20 and the external terminal 10. For example, the linkage control unit 203 sends the decryption information 52 stored in the database 204 to the linkage control unit 107 of the external terminal 10. As a result, the decryption information 52 is shared between the work terminal 20 and the external terminal 10, enabling coordination.

[External device]
The external terminal 10 is a computer that directly provides confidential information, which is not displayed on the work terminal 20, to the user performing the business negotiation or other tasks using voice or other means, when the user is conducting business negotiations or other tasks with customers using the work terminal 20. The external terminal 10 is, for example, a smartphone. The external terminal 10 is an example of the "second information processing device". As shown in Figure 1, the external terminal 10 has an image recognition unit 101, an image acquisition unit 102, a UI component coordinate identification unit 103, an ID identification unit 104, a decoding unit 105, an audio playback control unit 106, a cooperation control unit 107, an input/output interface 108, and a database 109.

The image acquisition unit 102, upon receiving an operation from the user, captures the image 210 displayed on the display device 201 of the work terminal 20. For example, the image acquisition unit 102 captures the image 210 using a camera (not shown) of an external terminal 10. Then, the image acquisition unit 102 outputs the acquired image 210 to the image recognition unit 101.

The image recognition unit 101 receives data from the image 210 acquired by the image acquisition unit 102. The image recognition unit 101 then recognizes the objects displayed in the image 210. The image recognition unit 101 then extracts UI components such as masking components 211 and ID markers 212 that have been added to the image 210. Subsequently, the image recognition unit 101 outputs information about the extracted UI components, such as the masking components 211 and ID markers 212, along with the image 210 data, to the UI component coordinate identification unit 103.

The UI component coordinate identification unit 103 receives data of the image 210 along with information about the UI components attached to the image 210 from the image recognition unit 101. Next, the UI component coordinate identification unit 103 identifies the coordinates of each UI component in the image 210 from the information about the UI components attached to the image 210. Then, the UI component coordinate identification unit 103 outputs the coordinate information of each UI component in the image 210, along with the data of the image 210, to the ID identification unit 104.

The ID identification unit 104 receives data for the image 210 and information on the coordinates of each UI component in the image 210 from the UI component coordinate identification unit 103. The ID identification unit 104 then uses the information on the coordinates of the UI components to analyze each UI component attached to the image 210 and determines whether an ID marker 212 or a masking component 211 containing the ID marker 212 exists. If neither an ID marker 212 nor a masking component 211 containing the ID marker 212 exists, the ID identification unit 104 outputs information indicating the absence of confidential information to the input/output interface 108 to notify the user.

In contrast, if an ID marker 212 or a masking component 211 containing the ID marker 212 exists, the ID identification unit 104 reads the ID from the ID marker 212. If the ID cannot be read, the ID identification unit 104 sends an error notification to the input/output interface 108 to notify the user of the error. Once the ID is obtained, the ID identification unit 104 outputs the ID read along with the ID marker 212 or the masking component 211 containing the ID marker 212 to the decoding unit 105.

This ID identification unit 104 is an example of a "reading unit." That is, the ID identification unit 104 reads the second UI component on the image 210 displayed on the screen of the first information processing device.

The decryption unit 105 receives the ID input from the ID identification unit 104 along with the ID marker 212 or the masking component 211 containing the ID marker 212. Next, the decryption unit 105 sends a request to the linkage control unit 107 to acquire the decryption information 52 corresponding to the ID. Subsequently, the decryption unit 105 receives the input of the decryption information 52 corresponding to the ID from the linkage control unit 107. If the decryption unit 105 receives a response from the linkage control unit 107 indicating that the acquisition of the ID failed because the decryption information 52 corresponding to the ID could not be acquired, the decryption unit 105 sends error information notifying the user of the failure to acquire the decryption information 52 to the input/output interface 108 to notify the user of the error. On the other hand, if the decryption information 52 corresponding to the ID is acquired, the decryption unit 105 uses the decryption information 52 to decrypt the encrypted confidential information contained in the ID marker 212 or the masking component 211 containing the ID marker 212.

If decryption fails, the decryption unit 105 sends error information indicating the decryption failure to the input/output interface 108 to notify the user of the error. Conversely, if decryption is successful, the decryption unit 105 acquires the confidential information. The decryption unit 105 then outputs the acquired confidential information to the audio playback control unit 106.

This decoding unit 105 is an example of an "extraction unit." The confidential information is an example of "predetermined information." In other words, the decoding unit 105 extracts information related to the predetermined information from the second UI component read by the reading unit. The decoding unit 105 also decodes the second UI component to obtain the predetermined information.

The linkage control unit 107 transmits and receives data with the linkage control unit 203 of the work terminal 20, and coordinates the external terminal 10 and the work terminal 20. For example, the linkage control unit 107 sends a request to the linkage control unit 203 of the work terminal 20 to acquire decryption information 52. Subsequently, the linkage control unit 107 receives the decryption information 52 from the linkage control unit 203 of the work terminal 20. Then, the linkage control unit 107 stores the received decryption information 52 in the database 109.

Furthermore, the linkage control unit 107 receives a request from the decryption unit 105 to acquire decryption information 52 corresponding to the ID. The linkage control unit 107 then acquires the decryption information 52 corresponding to the specified ID from the database 109. After that, the linkage control unit 107 transmits the acquired decryption information 52 to the decryption unit 105. If the decryption information 52 corresponding to the specified ID cannot be acquired, the linkage control unit 107 sends a response to the decryption unit 105 indicating that the acquisition of the decryption information 52 failed.

In this embodiment, the linkage control unit 107 pre-acquires decryption information 52 from the work terminal 20 and stores it in the database 109 of the external terminal 10, and then obtains decryption information 52 corresponding to the ID specified by the decryption unit 105 from the database 109. However, there are other methods for obtaining the decryption information 52. For example, the decryption information 52 may be pre-stored in the database 402 of a data storage device 40 other than the work terminal 20. In this case, the linkage control unit 107 can obtain the decryption information 52 from the linkage control unit 401 of the data storage device 40. Alternatively, without using the database 109 of the external terminal 10, the linkage control unit 107 may obtain the decryption information 52 corresponding to the ID specified by the linkage control unit 203 of the work terminal 20 each time it receives a request to obtain decryption information 52 from the decryption unit 105.

The audio playback control unit 106 is connected to an external audio output device 30, such as headphones. The audio playback control unit 106 receives confidential information as input from the decoding unit 105. If the confidential information is received as audio data, the audio playback control unit 106 plays back that audio data and outputs it as audio to the audio output device 30. If the confidential information is not in audio data format, the audio playback control unit 106 converts the confidential information into audio data, plays it back, and outputs it as audio to the audio output device 30. In other words, the audio playback control unit 106 plays back predetermined information extracted by the extraction unit as audio.

The input/output interface 108 has a display device such as a liquid crystal screen. When an error occurs, the input/output interface 108 receives an error notification from the ID identification unit 104 or the decoding unit 105. The input/output interface 108 then displays the error information notified by the error notification on the liquid crystal screen or the like to notify the user of the error. Furthermore, if the ID marker 212 or the masking component 211 containing the ID marker 212 is not present, the input/output interface 108 receives input from the ID identification unit 104 indicating the absence of confidential information. The ID identification unit 104 then provides the user with a message indicating the absence of confidential information, such as by displaying it on the liquid crystal screen.

The database 109 is a storage device for storing data. For example, the database 109 stores the decryption information 52 acquired by the cooperation control unit 107 from the work terminal 20.

[Operation overview]
Figure 2 is a diagram illustrating the general operation of the operation support system. Next, the overall operation of the operation support system 1 will be explained with reference to Figure 2.

In Figure 2, state 501 represents the work state using the work terminal 20 when the operation support system 1 is not used. When the operation support system 1 is not used, as shown in state 501, the display device 201 of the work terminal 20 displays an image 210 that shows confidential information. Therefore, in state 501, if the work terminal 20 is used to conduct business negotiations with a customer, the confidential information will be disclosed to the customer.

In contrast, state 502 is the working state using the work terminal 20 when the operation support system 1 is used. By using the operation support system 1, the confidential information that was displayed in the image 210 shown by the display device 201 in state 501 is concealed by the masking component 211 in state 502. This makes it possible to conceal confidential information from customers even when conducting business negotiations with customers using the work terminal 20. Furthermore, in state 502, since the masking component 211 does not contain the ID marker 212, the ID marker 212, which is an encrypted and registered version of the confidential information concealed by the masking component 211, is added to the image 210.

The user can read the ID marker 212 using an external terminal 10 such as a smartphone, and have the confidential information output as audio from an audio output device 30 such as headphones. This allows the user to check the confidential information without the customer knowing, without having to go to another room and check the information in a place where the customer cannot see, thereby improving work efficiency.

[Image display processing]
Figure 3 is a flowchart of the image display processing by the work terminal according to the first embodiment. Next, referring to Figure 3, the flow of the image display processing by the work terminal 20 according to this embodiment, including the masking of confidential information, will be described.

The display control unit 202 acquires the image 210 to be displayed. Then, the display control unit 202 uses the masking rules 51 stored in the database 204 to determine whether or not the image 210 is subject to masking (step S101).

If image 210 is not the object to be masked (step S101: negative), the display control unit 202 causes image 210 to be displayed on the display device 201 (step S102).

In contrast, if image 210 is to be masked (step S101: affirmative), the display control unit 202 acquires the masking component 211 according to the masking rule 51 (step S103).

Next, the display control unit 202 determines whether or not the acquisition of the masking component 211 was successful (step S104).

If the acquisition of the masking component 211 fails (step S104: negative), the display control unit 202 displays an error message on the display device 201 to notify the user of the error (step S105).

In contrast, if the acquisition of the masking component 211 is successful (step S104: affirmative), the display control unit 202 adds the masking component 211 to the image 210 to mask the confidential information (step S106).

Next, the display control unit 202 causes the image 210 with the masking component 211 added to be displayed on the display device 201 (step S107).

Next, the display control unit 202 determines whether or not an ID marker 212 corresponding to the masking component 211 exists, according to the masking rule 51 (step S108). If the ID marker 212 does not exist (step S108: negative), the display control unit 202 terminates the image display process. The case where the ID marker 212 does not exist also includes the case where the masking component 211 has an ID marker 212.

If ID marker 212 exists (step S108: affirmative), the display control unit 202 acquires ID marker 212 (step S109).

Subsequently, the display control unit 202 displays the acquired ID marker 212 near the masking component 211 in the image 210 (step S110).

[Confidential Information Acquisition Process]
Figure 4 is a flowchart of the confidential information acquisition process by an external terminal according to the first embodiment. Next, the flow of the confidential information acquisition process by the external terminal 10 according to this embodiment will be explained with reference to Figure 4.

The image acquisition unit 102 uses a camera or the like on the external terminal 10 to acquire the image 210 displayed on the display device 201 of the work terminal 20 (step S111). Then, the image acquisition unit 102 outputs the acquired image 210 to the image recognition unit 101.

The image recognition unit 101 recognizes objects displayed in the image 210 input from the image acquisition unit 102. The image recognition unit 101 then extracts UI components from the objects displayed in the image 210. The UI component coordinate identification unit 103 identifies the coordinates of each UI component extracted by the image recognition unit 101. The ID identification unit 104 determines whether or not an ID marker 212 exists among the UI components identified by the UI component coordinate identification unit 103 (step S112). If an ID marker 212 does not exist (step S112: negative), the confidential information acquisition process returns to step S111.

In contrast, if an ID marker 212 exists (step S112: affirmative), the ID identification unit 104 reads the ID marker 212 from the image 210 and performs ID identification (step S113). Here, the ID identification unit 104 reads the ID marker 212 not only from a single ID marker 212, but also from a masking component 211 containing the ID marker 212 if such a component exists.

The ID identification unit 104 determines whether the reading of the ID marker 212 was successful or not (step S114). If the reading of the ID marker 212 fails (step S114: negative), the ID identification unit 104 sends error information to the input/output interface 108 to notify the user of the error (step S118). After that, the confidential information acquisition process returns to step S111.

In response to this, if the reading of the ID marker 212 is successful (step S114: affirmative), the ID identification unit 104 outputs the acquired ID to the decoding unit 105. The decoding unit 105 sends a request to the linkage control unit 107 to acquire decoding information 52 corresponding to the ID acquired from the ID identification unit 104. Subsequently, based on the response from the linkage control unit 107, the decoding unit 105 determines whether or not decoding information 52 corresponding to the acquired ID exists (step S115).

If the decryption information 52 corresponding to the ID does not exist (step S115: negative), the decryption unit 105 sends error information to the input/output interface 108 to notify the user of the error (step S118). After that, the confidential information acquisition process returns to step S111.

In contrast, if decryption information 52 corresponding to the ID exists (step S115: affirmative), the decryption unit 105 obtains the decryption information 52 corresponding to the ID from the control unit. Then, the decryption unit 105 uses the obtained ID to decrypt the encrypted confidential information registered in the ID marker 212 (step S116).

The decryption unit 105 then determines whether or not the decryption was successful (step S117). If the decryption fails (step S117: negative), the decryption unit 105 sends error information to the input/output interface 108 to notify the user of the error (step S118). After that, the confidential information acquisition process returns to step S111.

In response to this, if the decryption is successful (step S117: affirmative), the decryption unit 105 outputs the confidential information obtained by decryption to the audio playback control unit 106. The audio playback control unit 106 plays the confidential information as audio and outputs the audio to the audio output device 30 (step S119).

[Effects of the first embodiment]
As described above, in the operation support system 1 according to this embodiment, the work terminal 20 conceals the confidential information in the image 210 on the screen by adding UI components for masking, and further adds an ID marker 212 to which the encrypted confidential information is registered. When a user reads the image 210 on the screen of the work terminal 20 using an external terminal 10, the external terminal 10 identifies the ID marker 212 from the read image 210, decrypts it to obtain the confidential information contained in the ID marker 212, and provides it to the user as audio.

This allows for the concealment of confidential information displayed on the work terminal 20 using UI extension technology, and enables the provision of confidential information only to specific individuals, such as the owner of the external terminal 10, while concealing it from others in the vicinity. Therefore, specific individuals can easily and confidentially verify the confidential information without the need to go to a separate room, thus ensuring the security of work involving images 210 containing confidential information while improving work efficiency.

[Second Embodiment]
[Configuration of the operation support system]
Figure 5 is a block diagram of the operation support system according to the second embodiment. In this embodiment, if the decryption information 52 corresponding to the ID of the ID marker 232 does not exist, the operation support system 1 directly obtains the confidential information corresponding to that ID from the database 109 or the like and outputs it as audio. The details of the operation support system 1 will be described below. In the following description, the operation of each part which is the same as in the first embodiment may be omitted from the explanation.

The work terminal 20 will now be described. The database 204 holds the masking rules 51 and decryption information 52, as well as the confidential information files 53 generated by the display control unit 202. Each confidential information file 53 is a file in which individual confidential information is registered and associated with the ID of the ID marker 212.

The display control unit 202 holds the confidential information masked by the masking component 211. It then generates a confidential information file 53 containing the confidential information corresponding to the ID marker 212 added to the image 210 or the ID of the ID marker 212 contained in the masking component 211, and stores it in the database 204.

The linkage control unit 203 retrieves the decryption information 52 and the confidential information file 53 stored in the database 204. Then, the linkage control unit 203 transmits the retrieved decryption information 52 and confidential information file 53 to the external terminal 10.

Next, the external terminal 10 will be described. In addition to the parts shown in the first embodiment, the external terminal 10 has a confidential information acquisition unit 110.

The decoding unit 105 reads the ID from the ID marker 212 attached to the image 210. This ID of the ID marker 212 is an example of "identification information attached to the second UI component." That is, the ID identification unit 104 reads the identification information attached to the second UI component. Then, if the decoding unit 105 does not have the decoded information 52 corresponding to the read ID in the database 109, it requests the confidential information acquisition unit 110 to retrieve the confidential information corresponding to the ID from the database 109.

The confidential information acquisition unit 110 receives a request from the decryption unit 105 to acquire confidential information from the database 109. The confidential information acquisition unit 110 then requests the cooperation control unit 107 to acquire the confidential information file 53 corresponding to the ID. Subsequently, the confidential information acquisition unit 110 receives a response to the acquisition request from the cooperation control unit 107.

If the confidential information file 53 is received as a response from the linkage control unit 107, the confidential information acquisition unit 110 acquires the confidential information from the acquired confidential information file 53. Then, the confidential information acquisition unit 110 outputs the acquired confidential information to the ID identification unit 104. On the other hand, if the linkage control unit 107 receives a response indicating the absence of the confidential information file 53, the confidential information acquisition unit 110 transmits information about the absence of confidential information to the input/output interface 108 to notify the user of the error.

This confidential information acquisition unit 110 is an example of an "acquisition unit." The confidential information file 53 is information in which identification information and predetermined information are associated. That is, the confidential information acquisition unit 110 acquires predetermined information corresponding to the identification information read by the reading unit from the database 109, where identification information and predetermined information are registered in association.

The audio playback control unit 106 receives confidential information as input from the confidential information acquisition unit 110. The audio playback control unit 106 then converts the acquired confidential information into audio data and transmits it to the audio output device 30, providing the confidential information to the user as audio.

The linkage control unit 107 receives the decryption information 52 and the confidential information file 53 from the linkage control unit 203 of the work terminal 20. Then, the linkage control unit 107 stores the acquired decryption information 52 and confidential information file 53 in the database 109.

Furthermore, the linkage control unit 107 receives a request from the confidential information acquisition unit 110 to acquire the confidential information file 53 along with the ID. The linkage control unit 107 then acquires the confidential information file 53 corresponding to the ID from the database 109. Finally, the linkage control unit 107 outputs the acquired confidential information file 53 to the confidential information acquisition unit 110.

In this embodiment, the linkage control unit 107 pre-acquired the decryption information 52 and the confidential information file 53 from the work terminal 20 and stored them in the database 109 of the external terminal 10. Then, the linkage control unit 107 acquired the decryption information 52 or the confidential information file 53 corresponding to the ID specified by the decryption unit 105 from the database 109. However, the method for acquiring both or either of the decryption information 52 and the confidential information file 53 may be by other means. For example, the decryption information 52 and the confidential information file 53 may be pre-stored in the database 402 of a data storage device 40 other than the work terminal 20. In this case, the linkage control unit 107 can acquire the decryption information 52 from the linkage control unit 401 of the data storage device 40. Alternatively, without using the database 109 of the external terminal 10, the linkage control unit 107 may acquire the decryption information 52 corresponding to the ID specified by the linkage control unit 203 of the work terminal 20 each time it receives a request to acquire the decryption information 52 from the decryption unit 105.

[Confidential Information Acquisition Process]
Figure 6 is a flowchart of the confidential information acquisition process by an external terminal according to the second embodiment. Next, the flow of the confidential information acquisition process by the external terminal 10 according to this embodiment will be explained with reference to Figure 6.

The image acquisition unit 102 uses a camera or the like on the external terminal 10 to acquire the image 210 displayed on the display device 201 of the work terminal 20 (step S201). Then, the image acquisition unit 102 outputs the acquired image 210 to the image recognition unit 101.

The image recognition unit 101 recognizes objects displayed in the image 210 input from the image acquisition unit 102. The image recognition unit 101 then extracts UI components from the objects displayed in the image 210. The UI component coordinate identification unit 103 identifies the coordinates of each UI component extracted by the image recognition unit 101. The ID identification unit 104 determines whether or not an ID marker 212 exists among the UI components identified by the UI component coordinate identification unit 103 (step S202). If an ID marker 212 does not exist (step S202: negative), the confidential information acquisition process returns to step S201.

In contrast, if an ID marker 212 exists (step S202: affirmative), the ID identification unit 104 reads the ID marker 212 from the image 210 and performs ID identification (step S203).

Next, the ID identification unit 104 determines whether or not the reading of the ID marker 212 was successful (step S204). If the reading of the ID marker 212 fails (step S204: negative), the ID identification unit 104 sends error information to the input/output interface 108 to notify the user of the error (step S208). After that, the confidential information acquisition process returns to step S201.

In response to this, if the reading of the ID marker 212 is successful (step S204: affirmative), the ID identification unit 104 outputs the acquired ID to the decoding unit 105. The decoding unit 105 sends a request to the linkage control unit 107 to acquire decoding information 52 corresponding to the ID acquired from the ID identification unit 104. Subsequently, based on the response from the linkage control unit 107, the decoding unit 105 determines whether or not decoding information 52 corresponding to the acquired ID exists (step S205).

If decryption information 52 corresponding to the ID exists (step S205: affirmative), the decryption unit 105 obtains the decryption information 52 corresponding to the ID from the control unit. Then, the decryption unit 105 uses the obtained ID to decrypt the encrypted confidential information registered in the ID marker 212 (step S206).

The decryption unit 105 then determines whether or not the decryption was successful (step S207). If the decryption fails (step S207: negative), the decryption unit 105 sends error information to the input/output interface 108 to notify the user of the error (step S208). After that, the confidential information acquisition process returns to step S201.

In response to this, if decryption is successful (step S207: affirmative), the decryption unit 105 outputs the confidential information obtained by decryption to the audio playback control unit 106. The audio playback control unit 106 plays the confidential information as audio and outputs the audio to the audio output device 30 (step S209).

On the other hand, if the decryption information 52 corresponding to the ID does not exist (step S205: negative), the decryption unit 105 requests the confidential information acquisition unit 110 to retrieve the confidential information corresponding to the ID from the database 109. Upon receiving the request from the decryption unit 105, the confidential information acquisition unit 110 sends a request to the cooperation control unit 107 to retrieve the confidential information corresponding to the ID. Subsequently, based on the response from the cooperation control unit 107, the confidential information acquisition unit 110 determines whether or not confidential information corresponding to the retrieved ID exists (step S210).

If no confidential information corresponding to the acquired ID exists (step S210: negative), the confidential information acquisition unit 110 sends error information to the input/output interface 108 to notify the user of the error (step S208). After that, the confidential information acquisition process returns to step S201.

In contrast, if confidential information corresponding to the acquired ID exists (step S210: affirmative), the confidential information acquisition unit 110 acquires the confidential information from the confidential information file 53 acquired from the cooperation control unit 107 (step S211).

The confidential information acquisition unit 110 then determines whether the acquisition of confidential information was successful (step S212). If the acquisition of confidential information fails (step S212: negative), the confidential information acquisition unit 110 sends error information to the input/output interface 108 to notify the user of the error (step S208). After that, the confidential information acquisition process returns to step S201.

In response to this, if the acquisition of confidential information is successful (step S212: affirmative), the confidential information acquisition unit 110 outputs the acquired confidential information to the audio playback control unit 106. The audio playback control unit 106 plays the confidential information as audio and outputs the audio to the audio output device 30 (step S209).

[Effects in the second embodiment]
As described above, in the operation support system 1 according to this embodiment, when it is difficult to decrypt the encrypted confidential information registered in the ID marker 212, the external terminal 10 directly obtains the confidential information from the database 109 and outputs it as audio data to the audio output device 30.

This ensures that even if the confidential information cannot be obtained from the ID marker 212 due to reasons such as the inability to obtain the decryption information 52, the confidential information can be obtained through other means and provided to a specific person, such as the owner of the external terminal 10, thus ensuring that the specific person can reliably obtain the confidential information. Therefore, it becomes possible to improve work efficiency more reliably while ensuring the security of work using the image 210 containing confidential information.

[Third Embodiment]
[Configuration of the operation support system]
Figure 7 is a block diagram of the operation support system according to the third embodiment. In the operation support system 1 according to this embodiment, the work terminal 20 can include query information that prompts the user to decide on an operation on the image 210 when encrypting and registering confidential information in the ID marker 212. The external terminal 10 plays the query information attached to the confidential information as audio and receives input of operation instructions for that query information from the user. The external terminal 10 then notifies the work terminal 20 of the input operation instructions, and the work terminal 20, upon receiving the notification, operates the image 210 according to the operation instructions. The details of the operation support system 1 will be described below. In the following description, operations similar to those of each part in the first and second embodiments may be omitted.

The work terminal 20 will now be described. In addition to the masking rules 51, decryption information 52, and confidential information file 53, the database 204 pre-stores display rules 54. Display rules 54 are generated for each ID marker 212 and include information such as the type of notification that is permitted to be accepted.

The display control unit 202 can include query information, such as choice information for the user to select and input request information for the user to input, when generating an ID marker 212 in which encrypted confidential information is registered. For example, choice information may be an option to select whether or not to remove the masking component 211 that masks the confidential information. Input request information may be information prompting the user to input characters to be displayed on the masking component 211 that masks the confidential information. The display control unit 202 determines whether or not to include query information in the ID marker 212 and the content of the query information, according to instructions from the administrator of the operation support system 1 or the like.

Furthermore, the display control unit 202 also includes inquiry information in the confidential information file 53, similar to the ID marker 212, in accordance with instructions from the administrator of the operation support system 1 or the like.

Here, we will explain the user's actions. When the user captures an image 210 using the external terminal 10 and obtains confidential information, if there is inquiry information corresponding to the confidential information, the user listens to the audio playback to confirm the inquiry information. The user can then instruct the work terminal 20 to perform an operation on the image 210 using the answer information, which includes the answer to the inquiry information. When answering the inquiry information, the user inputs an operation instruction for the image 210 using the touch panel or the like on the input/output interface 108 of the external terminal 10. Specifically, if the inquiry information is choice information, the user selects an operation instruction from the presented options, and if it is input request information, the user performs an operation instruction by inputting information according to the request.

When the display control unit 202 receives input of response information, which is a reply to the transmitted inquiry information, it obtains the response information from the linkage control unit 203. At this time, the ID of the ID marker 212 is added to the response information so that it is clear which ID marker 212 the response is to. The display control unit 202 then determines whether the ID included in the response information is an ID that has been registered as an ID that is authorized to accept operation instructions. If it is not an ID that has been registered, the display control unit 202 displays an error message on the display device 201 to notify the user.

In response to this, if the ID is registered, the display control unit 202 checks the display rule 54 corresponding to the ID marker 212 indicated by the ID included in the response information, and determines whether the type of notification of the received inquiry information matches the notification type described in the display rule 54. If the type of notification received does not match the notification type described in the display rule 54, the display control unit 202 displays an error message on the display device 201 to notify the user.

On the other hand, if the type of notification received matches the type of notification described in display rule 54, the display control unit 202 performs the operation instructed on the image 210 displayed on the display device 201 according to the received notification. For example, the display control unit 202 removes the masking component 211 from the image 210.

The linkage control unit 203 receives notification of response information to the inquiry information from the linkage control unit 107 of the external terminal 10. Then, the linkage control unit 203 outputs the response information to the display control unit 202.

Next, the external terminal 10 will be described. The decryption unit 105 uses the decryption information 52 corresponding to the ID of the ID marker 212 to decrypt the encrypted confidential information registered in the ID marker 212. Then, the decryption unit 105 plays the confidential information obtained by decryption as audio and transmits it to the audio playback control unit 106. At this time, if there is inquiry information attached to the ID marker 212, the decryption unit 105 transmits the inquiry information along with the confidential information to the audio playback control unit 106.

If the confidential information acquisition unit 110 does not have decryption information 52 corresponding to the ID of the ID marker 212, it acquires the confidential information file 53 corresponding to the ID acquired from the decryption unit 105 from the linkage control unit 107. Then, the confidential information acquisition unit 110 transmits the confidential information contained in the acquired confidential information file 53 to the audio playback control unit 106. At this time, if there is inquiry information attached to the ID marker 212, the confidential information acquisition unit 110 also acquires the inquiry information contained in the confidential information file 53 and transmits the inquiry information along with the confidential information to the audio playback control unit 106.

The audio playback control unit 106 receives the confidential information with the added query information from the decoding unit 105 or the confidential information acquisition unit 110. Then, the audio playback control unit 106 plays the query information along with the confidential information and outputs the audio to the audio output device 30. For example, the audio playback control unit 106 plays the query information after playing the confidential information and outputs the audio to the audio output device 30. In other words, the audio playback control unit 106 plays the query information, which prompts input related to the predetermined information, along with the predetermined information.

The operation information reflection unit 111 determines whether the audio playback of the confidential information sent from the decoding unit 105 or the audio playback of the confidential information sent from the confidential information acquisition unit 110 includes inquiry information. For example, the operation information reflection unit 111 may obtain information from the decoding unit 105 or the confidential information acquisition unit 110 regarding whether or not the inquiry information was played back. In this case, the operation information reflection unit 111 may also obtain the ID of the ID marker 212 corresponding to the transmitted confidential information. If the audio playback by the audio playback control unit 106 includes inquiry information, the operation information reflection unit 111 receives input of operation information corresponding to the inquiry information from the input/output interface 108. The operation information reflection unit 111 then sends a notification of audio response information including the acquired operation information and the ID of the ID marker 212 to the cooperation control unit 107, instructing it to send it to the work terminal 20.

In other words, the operation information reflection unit 111 receives the input of operation instructions in response to the inquiry information and notifies the display control unit 202 of the operation instructions. The display control unit 202 then reflects the operation instructions notified by the operation information reflection unit 111 in the image 210 displayed on the screen.

The linkage control unit 107 receives notification of response information from the operation information reflection unit 111. Then, in accordance with the instructions from the operation information reflection unit 111, the linkage control unit 107 transmits the notification of response information to the work terminal 20.

[Confidential Information Acquisition Process]
Figure 8 is a flowchart of the confidential information acquisition process by an external terminal according to the third embodiment. Next, the flow of the confidential information acquisition process by the external terminal 10 according to this embodiment will be explained with reference to Figure 8.

The image acquisition unit 102 uses a camera or the like on the external terminal 10 to acquire the image 210 displayed on the display device 201 of the work terminal 20 (step S301). Then, the image acquisition unit 102 outputs the acquired image 210 to the image recognition unit 101.

The image recognition unit 101 recognizes objects displayed in the image 210 input from the image acquisition unit 102. The image recognition unit 101 then extracts UI components from the objects displayed in the image 210. The UI component coordinate identification unit 103 identifies the coordinates of each UI component extracted by the image recognition unit 101. The ID identification unit 104 determines whether or not an ID marker 212 exists among the UI components identified by the UI component coordinate identification unit 103 (step S302). If an ID marker 212 does not exist (step S302: negative), the confidential information acquisition process returns to step S301.

In contrast, if an ID marker 212 exists (step S302: affirmative), the ID identification unit 104 reads the ID marker 212 from the image 210 and performs ID identification (step S303).

Next, the ID identification unit 104 determines whether or not the reading of the ID marker 212 was successful (step S304). If the reading of the ID marker 212 fails (step S304: negative), the ID identification unit 104 sends error information to the input/output interface 108 to notify the user of the error (step S308). After that, the confidential information acquisition process returns to step S301.

In response to this, if the reading of the ID marker 212 is successful (step S304: affirmative), the ID identification unit 104 outputs the acquired ID to the decoding unit 105. The decoding unit 105 sends a request to the linkage control unit 107 to acquire decoding information 52 corresponding to the ID acquired from the ID identification unit 104. Subsequently, based on the response from the linkage control unit 107, the decoding unit 105 determines whether or not decoding information 52 corresponding to the acquired ID exists (step S305).

If decryption information 52 corresponding to the ID exists (step S305: affirmative), the decryption unit 105 obtains the decryption information 52 corresponding to the ID from the control unit. Then, the decryption unit 105 uses the obtained ID to decrypt the encrypted confidential information registered in the ID marker 212 (step S306).

The decryption unit 105 then determines whether or not the decryption was successful (step S307). If the decryption fails (step S307: negative), the decryption unit 105 sends error information to the input/output interface 108 to notify the user of the error (step S308). After that, the confidential information acquisition process returns to step S301.

If the decryption is successful (step S307: affirmative), the decryption unit 105 outputs the confidential information obtained by decryption to the audio playback control unit 106. The audio playback control unit 106 plays the confidential information as audio and outputs the audio to the audio output device 30 (step S312).

On the other hand, if the decryption information 52 corresponding to the ID does not exist (step S305: negative), the decryption unit 105 requests the confidential information acquisition unit 110 to retrieve the confidential information corresponding to the ID from the database 109. Upon receiving the request from the decryption unit 105, the confidential information acquisition unit 110 transmits a request to the cooperation control unit 107 to retrieve the confidential information corresponding to the ID. Subsequently, based on the response from the cooperation control unit 107, the confidential information acquisition unit 110 determines whether or not confidential information corresponding to the retrieved ID exists (step S309).

If no confidential information corresponding to the acquired ID exists (step S309: negation), the confidential information acquisition unit 110 sends error information to the input/output interface 108 to notify the user of the error (step S308). After that, the confidential information acquisition process returns to step S301.

In contrast, if confidential information corresponding to the acquired ID exists (step S309: affirmative), the confidential information acquisition unit 110 acquires the confidential information from the confidential information file 53 acquired from the cooperation control unit 107 (step S310).

The confidential information acquisition unit 110 then determines whether the acquisition of confidential information was successful (step S311). If the acquisition of confidential information fails (step S311: negative), the confidential information acquisition unit 110 sends error information to the input/output interface 108 to notify the user of the error (step S308). After that, the confidential information acquisition process returns to step S301.

In response to this, if the acquisition of confidential information is successful (step S311: affirmative), the confidential information acquisition unit 110 outputs the acquired confidential information to the audio playback control unit 106. The audio playback control unit 106 generates audio from the confidential information and outputs the audio to the audio output device 30 (step S312).

After audio playback in step S312, the operation information reflection unit 111 determines whether or not audio playback of the inquiry information was performed in the audio playback of the confidential information sent from the decoding unit 105 or the audio playback of the confidential information sent from the confidential information acquisition unit 110 (step S313).

If the audio playback of the inquiry information is not performed (step S313: negative), the external terminal 10 terminates the confidential information acquisition process.

In response to this, if the inquiry information is played back audibly (step S313: affirmative), the operation information reflection unit 111 receives operation information corresponding to the inquiry information from the input/output interface 108. Then, the operation information reflection unit 111 transmits the acquired operation information and the response information including the ID of the ID marker 212 to the work terminal 20 via the cooperation control unit 107 (step S314).

[Image display processing]
Figure 9 is a flowchart of the image display processing by the work terminal according to the third embodiment. Next, referring to Figure 9, the flow of the image display processing, including the masking of confidential information and operations on the masking component 211 by the work terminal 20 according to this embodiment, will be described.

The display control unit 202 acquires the image 210 to be displayed. Then, the display control unit 202 uses the masking rules 51 stored in the database 204 to determine whether or not the image 210 is subject to masking (step S321).

If image 210 is not the target of masking (step S321: negative), the display control unit 202 displays image 210 on the display device 201 (step S322). After that, the work terminal 20 terminates the image display process.

In contrast, if image 210 is to be masked (step S321: affirmative), the display control unit 202 acquires the masking component 211 according to the masking rule 51 (step S323).

Next, the display control unit 202 determines whether or not the acquisition of the masking component 211 was successful (step S324).

If the acquisition of the masking component 211 fails (step S324: negative), the display control unit 202 displays an error message on the display device 201 to notify the user of the error (step S334).

In contrast, if the acquisition of the masking component 211 is successful (step S324: affirmative), the display control unit 202 adds the masking component 211 to the image 210 to mask the confidential information (step S325).

Next, the display control unit 202 causes the image 210 with the masking component 211 added to be displayed on the display device 201 (step S326).

Next, the display control unit 202 determines whether or not an ID marker 212 corresponding to the masking component 211 exists, according to the masking rule 51 (step S327). If the ID marker 212 does not exist (step S327: negative), the image display process proceeds to step S330.

In contrast, if an ID marker 212 exists (step S327: affirmative), the display control unit 202 acquires the ID marker 212 (step S328).

Subsequently, the display control unit 202 displays the acquired ID marker 212 near the masking component 211 in the image 210 (step S329).

Subsequently, the display control unit 202 determines whether or not it has received response information from the external terminal 10 (step S330).

If no response information has been received (step S330: negative), the work terminal 20 terminates the image display process.

In response to this, if the response information is received (step S330: affirmative), the display control unit 202 determines whether the ID of the ID marker 212 included in the response information is an ID that has been registered as an ID that is authorized to accept operation instructions (step S331).

If the ID is not registered (step S331: negative), the display control unit 202 displays an error message on the display device 201 to notify the user (step S334).

In contrast, if the ID is already registered (step S331: affirmative), the display control unit 202 checks the display rule 54 and determines whether the type of notification received matches the notification type described in the display rule 54 (step S332).

If the type of notification received does not match the type of notification described in display rule 54 (step S332: negative), the display control unit 202 displays an error message on the display device 201 to notify the user (step S334).

In response to this, if the type of notification received matches the type of notification described in display rule 54 (step S332: affirmative), the display control unit 202 operates the image 210 according to the operation instructions included in the response information to reflect the operation instructions (step S333).

[Effects of the third embodiment]
As described above, in the operation support system 1 according to this embodiment, the external terminal 10 plays the inquiry information audibly along with the confidential information when there is inquiry information corresponding to the confidential information. Subsequently, when the external terminal 10 receives input of answer information that is the answer to the inquiry information, it transmits the answer information to the work terminal 20 and reflects the operation specified in the answer information on the image 210.

This allows for operations on the image 210 displayed by the work terminal 20 to be performed according to the will of a specific person, such as the owner of the external terminal 10, when confidential information can be obtained, thereby improving usability. Therefore, it becomes possible to further improve work efficiency while ensuring the security of operations using the image 210 containing confidential information.

[System Configuration, etc.]
Here, the components of each device illustrated above are functional concepts and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those illustrated, and all or part of them can be functionally or physically distributed or integrated in any unit depending on various loads and usage conditions. Furthermore, each processing function performed by each device can be implemented, in whole or in any part, by a CPU (Central Processing Unit) and a program that is analyzed and executed by the CPU, or by hardware using wired logic.

Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified.

[program]
In one embodiment, the external terminal 10 and work terminal 20 included in the operation support system 1 can be implemented by installing a confidential information acquisition program and an image display program that perform the above-mentioned information processing as packaged software or online software on a desired computer. For example, by having the above-mentioned confidential information acquisition program and image display program run on the information processing device, the information processing device can be made to function as the external terminal 10 and work terminal 20. The information processing device referred to here includes desktop or notebook personal computers. In addition, the category of information processing device also includes mobile communication terminals such as smartphones, mobile phones and PHS (Personal Handy-phone System), and slate terminals such as PDA (Personal Digital Assistant).

Furthermore, the external terminal 10 and work terminal 20 included in the operation support system 1 can also be implemented as a confidential information processing server device and an image display processing server device, respectively, which use the terminal device used by the user as a client and provide services related to the confidential information acquisition processing and image display processing described above to the client. For example, the confidential information processing server device is implemented as a server device that provides a confidential information acquisition service, taking image 210 as input and performing confidential information acquisition processing. Similarly, the image display processing server device is implemented as a server device that provides an image display service, taking the data of image 210 as input and performing image display processing. In this case, the confidential information processing server device and the image display processing server device may be implemented as a web server, or they may be implemented as a cloud service that provides the confidential information acquisition processing and image display processing described above through outsourcing.

Figure 10 shows an example of a computer that executes a confidential information acquisition program and an image display program included in an operation support program. Computer 1000 has, for example, memory 1010 and a CPU 1020. Computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

Memory 1010 includes ROM (Read Only Memory) 1011 and RAM (Random Access Memory) 1012. ROM 1011 stores, for example, a boot program such as BIOS (BASIC Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, a mouse 1110 and a keyboard 1120. The video adapter 1060 is connected to, for example, a display 1130.

The hard disk drive 1090 stores, for example, the OS 1091, application programs 1092, program modules 1093, and program data 1094. That is, operation support programs, including confidential information acquisition programs and image display programs that define the various processes of the external terminal 10 and the work terminal 20, which have functions equivalent to those of the external terminal 10 and the work terminal 20, are implemented as program modules 1093 in which executable code is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, a program module 1093 for performing processes similar to those in the external terminal 10 and the work terminal 20 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

Furthermore, the configuration data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes the processing of the above-described embodiment.

Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070.

1 Operation support system 10 External terminal 20 Work terminal 30 Audio output device 40 Data storage device 51 Masking rules 52 Decoded information 53 Confidential information file 54 Display rules 101 Image recognition unit 102 Image acquisition unit 103 UI component coordinate identification unit 104 ID identification unit 105 Decoder unit 106 Audio playback control unit 107 Coordination control unit 108 Input/output interface 109 Database 110 Confidential information acquisition unit 111 Operation information reflection unit 201 Display device 202 Display control unit 203 Coordination control unit 204 Database 210 Image 211 Masking component 212 ID marker 401 Coordination control unit 402 Database

Claims (7)

An operation support system having a first information processing device and a second information processing device,
The first information processing device is
The system includes a display control unit that masks predetermined information contained in an image using a first UI component, and displays the masked image on the screen together with a second UI component related to the predetermined information.
The aforementioned second information processing device is
A reading unit that reads the second UI component on the image displayed on the screen of the first information processing device,
An extraction unit extracts information related to the predetermined information from the second UI component read by the reading unit,
A voice playback control unit plays back the predetermined information and inquiry information related to the predetermined information based on the information extracted by the extraction unit , and
An operation support system characterized by comprising: an operation information reflection unit that receives input of operation instructions for the aforementioned inquiry information, notifies the operation instructions to the display control unit, and causes the display control unit to reflect the operation instructions on the image displayed on the screen . The display control unit adds the encrypted predetermined information to the second UI component,
The extraction unit decodes the second UI component to obtain the predetermined information,
The operation support system according to claim 1, characterized in that the audio playback control unit plays back the predetermined information as audio. The display control unit adds identification information to the second UI component,
The reading unit reads the identification information added to the second UI component,
The second information processing device further includes an acquisition unit that acquires the predetermined information corresponding to the identification information read by the reading unit from a database in which the identification information and predetermined information are registered in association.
The operation support system according to claim 1 or 2. The operation support system according to claim 1, characterized in that the display control unit masks the image using the first UI component, which includes the second UI component. A reading unit reads a second UI component from an image displayed on the screen of another information processing device , wherein predetermined information in the image is masked by a first UI component and the image includes a second UI component related to the predetermined information.
An extraction unit extracts information related to the predetermined information from the second UI component read by the reading unit,
A voice playback control unit plays back the predetermined information and inquiry information related to the predetermined information based on the information extracted by the extraction unit , and
An operation support device characterized by comprising: an operation information reflection unit that receives input of operation instructions for the aforementioned inquiry information, notifies the aforementioned operation instructions to the other information processing device, and causes the other information processing device to reflect the aforementioned operation instructions on the image displayed on the screen . The first information processing device masks predetermined information contained in an image using a first UI component, and displays the masked image on the screen together with a second UI component related to the predetermined information.
The second information processing device reads the second UI component on the image displayed on the screen of the first information processing device, extracts information related to the predetermined information from the read second UI component, plays audible inquiry information prompting input of the predetermined information and related information based on the extracted information related to the predetermined information , accepts input of operation instructions for the inquiry information, notifies the first information processing device of the operation instructions, and causes the first information processing device to reflect the operation instructions on the image displayed on the screen.
An operation support method characterized by the following. An operation support program characterized by: having a first computer perform a process of masking predetermined information contained in an image with a first UI component, displaying the masked image on a screen together with a second UI component related to the predetermined information; reading the second UI component on the image displayed on the screen, extracting information related to the predetermined information from the read second UI component , playing back the predetermined information and query information prompting input related to the predetermined information based on the extracted information, receiving input of operation instructions for the query information, notifying the first computer of the operation instructions, and having the second computer perform a process of reflecting the operation instructions on the image displayed on the screen .