JP7690274B2 - Information processing device, information processing method, and program - Google Patents

Description

The present disclosure relates to an information processing device, an information processing method, and a program.

In recent years, with the development of network technology and virtualization technology, it has become possible to use various functions by accessing services on a network, known as cloud services, without installing software on an information processing device such as a PC. One example of such a cloud service is cloud storage, which allows the creation, editing, viewing, and sharing of electronic documents on a network. Some cloud storage services also allow the viewing of editing history, the restoration of any version, and the extraction of differences between any versions.

Furthermore, files managed on the cloud can be viewed and edited regardless of the location where the user is working or the type of terminal device used for the work, as long as the cloud storage can be accessed via a network. Due to these characteristics, files managed on the cloud can also be shared in real time with other users in remote locations, for example.

On the other hand, even in recent years, confirmation and revision of the contents of electronic documents are often performed based on a printout of the contents of the electronic document on a recording medium such as paper. Against this background, various printers and MFPs (Multifunction Peripherals) that can print electronic documents managed in cloud storage by linking with the cloud storage have been proposed in recent years. For example, Patent Document 1 discloses an example of an image processing device that can print electronic documents managed in cloud storage.

In recent years, image processing such as OCR (Optical Character Recognition) has made it possible to recognize characters in an image and convert them into text data. Against this background, research is being conducted into a technology that recognizes the proofreading results reflected in a printed copy of an electronic document by performing image processing on the image obtained by reading the printed copy of the contents of the electronic document with a scanner or the like, and then reflects the results in the electronic document.

JP 2017-76244 A

On the other hand, characters added by hand to printed material tend to have different shapes depending on the writer. Due to this characteristic, when handwritten characters are recognized using image processing such as OCR, the accuracy of the recognition is lower than when printed characters are recognized, and as a result, situations may occur in which the target characters are recognized incorrectly. Therefore, for example, in a situation in which the proofreading results recognized by image processing of an image corresponding to the results of reading a printed material are reflected in an electronic document, even if a recognition error occurs, it is conceivable that the result of the recognition error will be reflected as is in the electronic document.

In consideration of the above problems, the present invention aims to make it possible to reflect the results of proofreading in an electronic document in a more suitable manner, even in a situation where the electronic document is proofread by adding edits to a printed matter on which the contents of the electronic document are printed.

The information processing device according to the present invention includes a scanner unit that reads a printed matter to which information indicating the location on cloud storage where the original electronic document is stored is attached as an image to generate a first image, and an operation unit that displays a screen for receiving from the user a first instruction specifying whether or not to reflect correction instructions written on the printed matter in the original electronic document stored in the location indicated by the information , a second instruction specifying a cloud storage to which the first image is to be sent from a list of cloud storages available to the user, and a third instruction specifying whether or not to add the correction instructions in a format conforming to a proofreading function of a document editing application capable of editing the original electronic document. The information processing device includes a receiving unit that receives the first image, the first instruction, the second instruction, and the third instruction from a device including the scanner unit that reads a printed matter to which information indicating the location on cloud storage where the original electronic document is stored is attached as an image to generate a first image, and an operation unit that displays a screen for receiving from the user a first instruction specifying whether or not to reflect correction instructions written on the printed matter in the original electronic document stored in the location indicated by the information, a second instruction specifying a cloud storage to which the first image is to be sent from a list of cloud storages available to the user, and a third instruction specifying whether or not to add the correction instructions in a format conforming to a proofreading function of a document editing application capable of editing the original electronic document. a transmission means for transmitting the first image received by the receiving means to the cloud storage specified in the second instruction, when the first instruction received by the receiving means specifies that the modification instructions should not be reflected in the original electronic document and the second instruction specifies a cloud storage to which the first image is to be transmitted; a generation means for generating a second image by rendering the original electronic document and generating a difference image by extracting a difference between the first image received by the receiving means and the generated second image, when the first instruction received by the receiving means specifies that the modification instructions should be reflected in the original electronic document and the original electronic document can be obtained from a location on the cloud storage based on information indicating a location on the cloud storage where the original electronic document is stored, the information being obtained by analyzing the first image received by the receiving means; and a specifying means for specifying a character string in the electronic document that is to be reflected by the correction instruction based on the coordinates of the proofreading symbol by performing analysis processing and OCR processing on the difference image obtained by the analysis processing and the OCR processing, and when the first instruction received by the receiving means specifies that the correction instruction should be reflected in the original electronic document, and based on the information indicating a location on the cloud storage where the original electronic document is stored, which is obtained by analyzing the first image received by the receiving means, the original electronic document can be acquired from that location on the cloud storage, and when the third instruction specifies that the correction instruction should be added in a format conforming to the proofreading function of the document editing application, the correction content based on the correction instruction is stored in the cloud storage and the document editing application is edited based on the correction instruction. and an execution means for executing a process of adding the correction content based on the correction instruction to the character string in the electronic document identified by the identification means as a record of the change history in the proofreading function, and when the first instruction received by the receiving means specifies that the correction instruction should be reflected in the original electronic document, and the original electronic document can be obtained from the location on the cloud storage where the original electronic document is stored based on the information obtained by analyzing the first image received by the receiving means and indicating the location on the cloud storage where the original electronic document is stored, and when the third instruction specifies that the correction instruction should not be added in a format conforming to the proofreading function of the document editing application, the execution means for executing a process of reflecting the correction content based on the correction instruction to the character string in the electronic document identified by the identification means without recording it as a change history in the proofreading function of the document editing application.

According to the present invention, even in a situation where an electronic document is proofread by adding to a printed matter on which the contents of the electronic document are printed, it is possible to add and reflect the results of the proofreading in the electronic document in a more suitable manner.

FIG. 1 illustrates an example of a system configuration of an information processing system. FIG. 2 illustrates an example of a hardware configuration of an MFP. FIG. 2 is a diagram illustrating an example of a software configuration of the MFP. FIG. 2 illustrates an example of a software configuration of an MFP server. FIG. 11 is a diagram showing an example of a menu screen. FIG. 13 is a diagram showing an example of a cloud print screen. FIG. 13 is a diagram showing an example of a cloud scan screen. 11 is a flowchart showing an example of processing of an information processing system. 10 is a flowchart showing another example of the processing of the information processing system.

A preferred embodiment of the present disclosure will be described in detail below with reference to the attached drawings. Note that in this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals to avoid redundant description.

<System Configuration>
An example of a system configuration of an information processing system according to the present embodiment will be described with reference to Fig. 1. The information processing system according to the present embodiment includes an MFP 100, a terminal device 113, an MFP server 120, and a cloud storage 121. The MFP 100, the terminal device 113, the MFP server 120, and the cloud storage 121 are connected to each other via a predetermined network.
1, the MFP 100 and the terminal device 113 are connected via a LAN 110. The MFP 120 and the cloud storage 121 are connected to the Internet 112. In addition, the communication via the LAN 110 and the communication via the Internet 112 are mediated by a gateway 111. This allows the MFP 100 and the terminal device 113 connected to the LAN 110 to access the MFP server 120 and the cloud storage 121.

Note that the configuration shown in FIG. 1 is merely an example, and the network configuration is not limited as long as it is possible to interconnect the MFP 100, the terminal device 113, the MFP server 120, and the cloud storage 121. In addition, the network connecting the various components described above, such as the LAN 110 and the Internet 112, may include multiple networks, and the type of some of the multiple networks may be different from the type of the other networks.

The MFP 100 includes a controller unit 101 , an operation unit 102 , a printer unit 103 , and a scanner unit 104 .
The operation unit 102 is a schematic representation of an interface for receiving instructions from a user for the MFP 100 and presenting various pieces of information to the user.
The printer unit 103 prints an image corresponding to electronic data (e.g., an electronic document, etc.) on a recording medium such as paper (hereinafter also referred to as a "sheet"), and outputs the printed matter with the image printed on the recording medium to a specified paper output destination.
The scanner unit 104 reads a printed material and converts the result of the reading (in other words, the result of optically reading the printed material) into an image signal to generate image data.
The controller unit 101 is connected to an operation unit 102, a printer unit 103, and a scanner unit 104, and controls the operations of these units, thereby achieving various functions of the MFP 100 (i.e., functions as a multifunction peripheral).

The terminal device 113 is a schematic representation of a print client that issues print instructions to the MFP 100 using a print protocol such as the Line Printer Remote (LPR) protocol. The terminal device 113 may also provide a user interface, such as a web browser, that accesses other devices (e.g., the MFP server 120 or cloud storage 121) via a network and enables the user to operate the other devices. The terminal device 113 may be realized, for example, by a PC (Personal Computer) or the like.

The gateway 111 corresponds to a so-called network router that mediates communication between a plurality of devices connected to different networks (in other words, relays communication).
The MFP server 120 is a server that executes various processes related to improving the functionality of the MFP 100 .
The cloud storage 121 is a so-called file sharing server that transmits and receives files (data) to and from the MFP 100 using a predetermined protocol such as HTTP (Hyper Text Transfer Protocol).

<Hardware Configuration>
An example of the hardware configuration of the MFP 100 will be described with reference to FIG. 2, focusing in particular on the configuration of the controller unit 101.
The controller unit 101 of the MFP 100 includes a central processing unit (CPU) 201, a dynamic random access memory (DRAM) 202, and an I/O controller 203. The controller unit 101 also includes a network I/F 204, a serial advanced technology attachment (SATA) 205, a panel I/F 206, a printer I/F 207, and a scanner I/F 208. The controller unit 101 may also include a universal serial bus (USB) 212.

The CPU 201 and the DRAM 202 are connected via a bus. The DRAM 202 is used as a working area for temporarily storing program data representing arithmetic instructions, data to be processed, and the like, in the process of the CPU 201 performing various arithmetic operations.
The CPU 201 and the I/O controller 203 are connected via a bus. The I/O controller 203 controls input/output to and from various devices in accordance with instructions from the CPU 201.

The I/O controller 203 is connected to a SATA I/F 205, which is connected to a storage device such as a Flash ROM 211. The Flash ROM 211 stores programs that the CPU 201 uses to realize the functions of the MFP 100, as well as various data such as document files.

Furthermore, a network I/F 204 is connected to the I/O controller 203, and a communication device for realizing communication with a predetermined network, such as a wired LAN device 210, is connected to the network I/F 204. The CPU 201 controls a communication device, such as the wired LAN device 210, via the network I/F 204, thereby realizing communication with other devices via a network, such as the LAN 110.

The I/O controller 203 is also connected to a panel I/F 206. The CPU 201 controls the operation of the operation unit 102 via the panel I/F 206, thereby accepting instructions from a user via the operation unit 102 and presenting various information to the user via the operation unit 102.

A printer I/F 207 is also connected to the I/O controller 203. The CPU 201 controls the operation of the printer unit 103 via the printer I/F 207, thereby realizing a process of printing an image on a recording medium (sheet) such as paper and outputting the printed matter to a predetermined paper output destination.
A scanner I/F 208 is also connected to the I/O controller 203. The CPU 201 controls the operation of the scanner unit 104 via the scanner I/F 208 to realize a process of reading an original (printed material) and converting the result of the reading into an image signal to generate image data.

A USB I/F 212 may also be connected to the I/O controller 203. The CPU 201 controls the operation of various devices connected to the USB I/F 212 via the USB I/F 212. An example of a device connected to the USB I/F 212 is an IC card reader 213. In this case, the CPU 201 controls the operation of the IC card reader 213 via the USB I/F 212 to realize, for example, processing related to user authentication using an IC card.

The CPU 201 executes processes related to the realization of various functions of the MFP 100 by controlling the operation of various devices as exemplified above. Here, an example of control when realizing a copy function is described below.

The CPU 201 reads out predetermined program data from the flash ROM 211 via the SATA I/F 205, and loads the data in the DRAM 202. The CPU 201 controls the operation of the operation unit 102 via the panel I/F 206 in accordance with the program loaded in the DRAM 202, thereby accepting a copy-related instruction from a user via the operation unit 102.
When the CPU 201 receives an instruction related to copying, it controls the operation of the scanner unit 104 via the scanner I/F 208 to cause the scanner unit 104 to read an original document and obtain image data (electronic data) according to the result of the reading. The CPU 201 stores the obtained image data in a predetermined storage area (e.g., DRAM 202).
The CPU 201 transfers the image data stored in a predetermined storage area to the printer unit 103 via the printer I/F 207, and instructs the printer unit 103 to execute processing related to printing based on the image data. In response to this instruction, the printer unit 103 prints an image corresponding to the image data on a recording medium (sheet) such as paper, and ejects the printed matter corresponding to the printing result to a predetermined paper ejection destination.

<Software configuration>
An example of the software configuration of the information processing system according to the present embodiment will be described below with reference to FIGS. 3 and 4, focusing in particular on the MFP 100 and the MFP server 120. FIG.

(Software configuration of MFP 100)
First, an example of the software configuration of the MFP 100 will be described with reference to FIG.
Components related to realizing the functions of the MFP 100 include, for example, an operation control unit 301, a data storage unit 302, a job control unit 303, an image processing unit 304, a print processing unit 305, and a reading processing unit 306. Components related to realizing the functions of the MFP 100 include a network control unit 307, a TCP/IP control unit 308, an authentication control unit 309, and an MFP server communication unit 310. The processing of each of the series of components shown in Fig. 3 is realized, for example, by the CPU 201 expanding a program stored in a storage area such as the Flash ROM 211 into a working area such as the DRAM 202 and executing the program.

The operation control unit 301 causes the operation unit 102 to display a screen image for accepting instructions from the user and presenting various information to the user, and controls the detection of user operations on the screen image and the execution of processes linked to components such as buttons displayed on the screen.

The data storage unit 302 stores various data in a storage area such as the Flash ROM 211, and executes processes related to reading out various data from the storage area. As a specific example, a case will be considered where a user issues an instruction to change device settings related to the operation of the MFP 100 via the operation unit 102. In this case, the operation control unit 301 recognizes the content of the instruction from the user accepted by the operation unit 102, and requests the data storage unit 302 to store the setting values of the device settings according to the content of the instruction. Upon receiving the request from the operation control unit 301, the data storage unit 302 stores the setting values according to the instruction from the user in a specified storage area (e.g., the Flash ROM 211, etc.) as the setting values of the device settings.

A job control unit 303 performs various controls related to the execution of a job.
The image processing unit 304 processes the image data into data of a desired format (for example, data in a format according to the intended purpose) in accordance with instructions from the job control unit 303 .
The print processing unit 305 controls the operation of the printer unit 103 via the printer I/F 207 in accordance with instructions from the job control unit 303, and performs various processes related to printing images on a recording medium (sheet) such as paper, and outputting printed matter to a specified paper output destination.
The reading processing unit 306 controls the operation of the scanner unit 104 via the scanner I/F 208 in accordance with instructions from the job control unit 303 , and executes various processes related to reading a document placed on the reading surface of the scanner unit 104 .

The network control unit 307 controls network-related settings such as IP addresses at specific times, such as when the system is started or when a setting change is detected, in accordance with setting values stored in a specific storage area, thereby controlling settings related to the operation of components related to communication, such as the TCP/IP control unit 308.
A TCP/IP control unit 308 executes various processes for transmitting and receiving network packets to and from other devices through the network to which the MFP 100 is connected via the network I/F 204 .

The authentication control unit 309 executes processing related to authentication of a user based on a predetermined authentication method. For example, the authentication control unit 309 may authenticate a user who uses the MFP 100 by comparing information read from an IC card by the IC card reader 213 with data related to the user stored in a predetermined storage area by the data storage unit 302.

The MFP server communication unit 310 causes the network control unit 307 to control communication with the MFP server 120 according to instructions from the job control unit 303.

Here, an example of the operation of each component when implementing a copy function will be described below.
For example, when the operation control unit 301 receives an instruction to start copying from a user, it instructs the job control unit 303 to start processing to realize a copy function. The job control unit 303 instructs the reading processing unit 306 to read an original document, and obtains image data generated based on an image signal output according to the result of the reading. In the following description, for convenience, the image according to the result of reading the original document is also referred to as a "scanned image."
The job control unit 303 instructs the image processing unit 304 to convert the acquired image data into data in a format suitable for printing the scanned image.
Then, the job control unit 303 instructs the print processing unit 305 to print based on the image data whose format has been converted by the image processing unit 304. In response to this instruction, the print processing unit 305 causes the printer unit 103 to execute processing related to printing based on the image data specified by the job control unit 303. As a result, the copy result of the original is output as a printed matter.

(Software configuration of MFP server 120)
Next, an example of the software configuration of the MFP server 120 will be described with reference to FIG.
Components related to realizing the functions of the MFP server 120 include, for example, a web browser communication unit 401, a data storage unit 402, a job control unit 403, an image processing unit 404, a cloud storage communication unit 405, an MFP communication unit 406, and an authentication control unit 407.
A series of software programs executed by the MFP server 120 may be held as distributed resources on a cloud, for example. Then, the programs may be loaded into a non-volatile memory allocated as the distributed resource, and then executed by a computing resource allocated as the distributed resource, thereby realizing the processing of each piece of software.

The web browser communication unit 401 executes processes related to presenting various information to the user and accepting instructions from the user in response to requests from a web browser operating on another device (e.g., the terminal device 113). The web browser communication unit 401 also controls the execution of processes linked to components such as buttons on a screen presented to the user via the web browser.

The data storage unit 402 executes processes related to storing various data in a storage area such as a distributed resource on the cloud and reading out various data from the storage area. As a specific example, let us consider a case where an instruction to change device settings related to the operation of the MFP server 120 is given via a web browser. In this case, the web browser communication unit 401 recognizes the content of the instruction from the user received by the web browser, and requests the data storage unit 402 to store the setting values of the above device settings according to the content of the instruction. Upon receiving the request from the web browser communication unit 401, the data storage unit 402 stores the setting values according to the instruction from the user in a specified storage area (e.g., a distributed resource on the cloud) as the setting values of the above device settings.

A job control unit 403 performs various controls related to the execution of a job.
The image processing unit 404 processes the image data into data of a desired format (for example, data in a format according to the intended purpose) in accordance with instructions from the job control unit 403 .
The cloud storage communication unit 405 controls communication with the cloud storage 121 by using the user's account information stored in a predetermined storage area by the data storage unit 402 .

The MFP communication unit 406 executes various processes related to communication with the MFP 100. As a specific example, the MFP communication unit 406 may execute processes related to accepting requests sent from the MFP 100, notifying the MFP 100 of various information, etc.

The authentication control unit 407 executes processing related to user authentication based on a predetermined authentication method. For example, the authentication control unit 407 may authenticate a user who uses the MFP server 120 by comparing the authentication information accepted by the web browser communication unit 401 with data related to the user stored in a predetermined storage area by the data storage unit 402.

<User Interface>
An example of a user interface of the information processing system according to this embodiment will be described with reference to FIGS.

(Menu screen)
First, Fig. 5 will be described. Fig. 5 is a diagram showing an example of a menu screen 501 displayed on the operation unit 102 of the MFP 100. The menu screen 501 is a screen for receiving instructions from a user relating to the execution of various functions possessed by the MFP 100. When the IC card reader 213 recognizes an IC card presented by a user, the authentication control unit 309 authenticates the target user based on information read from the IC card, and if the authentication is successful, the menu screen 501 is displayed.

The button 502 is a user interface for receiving an instruction from the user regarding the execution of the copy function.
The button 503 is a user interface for receiving instructions from the user regarding the execution of a print function (hereinafter also referred to as a “cloud print function”) that uses data (e.g., electronic documents) stored in the cloud storage 121 .
Button 504 is a user interface for receiving instructions from a user to execute a function (hereinafter also referred to as the “cloud scan function”) of reading a document and storing data corresponding to the results of the reading in cloud storage 121.

(Cloud print screen)
Next, Fig. 6 will be described. Fig. 6 is a diagram showing an example of a cloud print screen 601 displayed on the operation unit 102 of the MFP 100. The cloud print screen 601 is a screen for receiving instructions from a user regarding various settings of a print function (cloud print function) that uses data stored in the cloud storage 121. The cloud print screen 601 is displayed on the operation unit 102 when, for example, the button 503 is pressed on the menu screen 501.

The display area 602 is an area where a list of files available as targets for cloud printing (in other words, candidates for print targets) is displayed. The list of files displayed in the display area 602 may be displayed in a scrollable manner, for example, by a scroll bar. When a file presented in the list is selected, the file is set as a target for cloud printing.
It should be noted that the candidates for print targets displayed in the display area 602 may be limited to files linked to the account information of the user identified by authentication.
Furthermore, when multiple cloud storages are used as the cloud storage 121, for example, files stored in each of the multiple cloud storages may be displayed as a common list in the display area 602. In this case, information about the cloud storage in which each file is stored (for example, the cloud storage name, etc.) may be displayed in association with each file.

Check box 603 is a user interface for receiving an instruction from the user as to whether or not to associate information for identifying a path to a file selected to be printed (hereinafter also referred to as "URL information") with the file. By associating this URL information with a file, it becomes possible for another device, such as MFP 100, to identify the file (the file that is the source of a copy of the received file) managed by cloud storage 121 based on the URL information.

Button 604 is a user interface for receiving instructions from the user relating to the execution of the cloud print function in response to the user instructions received by display area 602 and check box 603. When button 604 is pressed, information corresponding to the user instructions received by display area 602 and check box 603 is sent to MFP server 120, and the MFP server 120 is instructed to execute various processes relating to the realization of the cloud print function.

The display area 605 is a display area where notification information (e.g., messages, etc.) for notifying the user of various situations is displayed. As a specific example, when an error occurs and the execution of a printing process is interrupted, information about the error may be displayed in the display area 605, thereby providing the user with feedback on the situation at that time.

(Cloud scan screen)
Next, Fig. 7 will be described. Fig. 7 is a diagram showing an example of a cloud scan screen 701 displayed on the operation unit 102 of the MFP 100. The cloud scan screen 701 is a screen for receiving instructions from a user regarding various settings of a function (cloud scan function) for reading an original and storing data corresponding to the results of the reading in the cloud storage 121. The cloud scan screen 701 is displayed on the operation unit 102 when, for example, the button 504 is pressed on the menu screen 501.

The display area 702 is an area where a list of cloud storages available as a destination for a file corresponding to the document reading result (e.g., a file of a scanned image) is displayed. The list of cloud storages displayed in the display area 702 may be displayed in a scrollable manner, for example, using a scroll bar. When a cloud storage presented in the list is selected, the cloud storage is set as a destination for a file corresponding to the document reading result.
A list of cloud storages available to a user identified by authentication may be displayed in the display area 702. In this case, for example, the MFP 100 may refer to information related to a connection to cloud storage for each user registered in the MFP server 120 to identify the list of cloud storages available to the user.

Check box 703 is a user interface for receiving an instruction from the user as to whether or not to use the URL information associated with the file selected as the print target when executing the cloud print described above for processing according to the scan results. By checking check box 703, for example, based on the URL information, cloud storage to which the file corresponding to the document reading results will be sent is specified, and the scan results are reflected in the file to be printed.

Check box 706 is a user interface for receiving instructions from the user regarding the method of reflecting information according to the results of reading the manuscript in the original file from which the manuscript was printed. For example, by checking check box 706, the information according to the results of reading the manuscript is added as proofreading information without being overwritten to the original file from which the manuscript was printed. In this disclosure, proofreading information corresponds to information indicating that the corrections were made when operating in an operation mode related to proofreading an electronic document in a specific software (document editing application) having a function related to editing the target electronic document. In addition, when check box 706 is not checked, the information according to the results of reading the manuscript is reflected by overwriting the original file from which the manuscript was printed.

Button 704 is a user interface for receiving instructions from the user to execute the cloud scan function in response to the user instructions received by display area 702 and check box 703. When button 704 is pressed, the document is read under the control of the reading processing unit 306, and data (file) corresponding to the results of the reading is sent to the MFP server 120. At this time, information corresponding to the user instructions received by display area 702, check box 703, and check box 706 is sent to the MFP server 120 in association with the data corresponding to the results of the reading of the document.

The display area 705 is a display area where notification information (e.g., messages, etc.) for notifying the user of various situations is displayed. As a specific example, when an error occurs and the execution of the process related to reading the document is interrupted, information related to the error may be displayed in the display area 705, thereby providing the user with feedback on the situation at that time.

<Processing>
An example of the processing of the information processing system according to the present embodiment will be described with particular focus on the cloud scan function with reference to Fig. 8. The series of processing shown in Fig. 8 is started, for example, by pressing the button 704 on the cloud scan screen 701 shown in Fig. 7.

In S801, the operation control unit 301 registers the processing content of the cloud scan in the job control unit 303 according to the selection state of the cloud storage in the display area 702 of the cloud scan screen 701 and the input states of the check boxes 703 and 706. Note that the selection state of the cloud storage in the display area 702 and the input states of the check boxes 703 and 706 may be registered as attributes of the processing content, for example.
The job control unit 303 requests the reading processing unit 306 to execute processing related to reading the document placed on the scanner unit 104 in accordance with the registered processing content. In response to this request, the reading processing unit 306 controls the operation of the scanner unit 104 to execute processing related to reading the document placed on the reading surface of the scanner unit 104.
The scanned image obtained as a result of reading the document is output, for example, as bitmap data (file) in which information obtained by optically reading the document is arranged two-dimensionally.

In S802, the MFP server communication unit 310 transmits data corresponding to the result of reading the document in S801 (i.e., data of the scanned image) as a processing request to the MFP server 120. At this time, information regarding the cloud storage selected in the display area 702 and the input states of the check boxes 703 and 706 are also notified to the MFP server 120.
Then, the MFP server communication unit 310 determines whether or not the transmission of the data corresponding to the result of reading the document to the MFP server 120 has been successful.
If the MFP server communication unit 310 determines in S802 that the transmission of data corresponding to the document reading result to the MFP server 120 has failed, the process proceeds to S820. In S820, the operation control unit 301 of the MFP 100 displays information about the error that has occurred in the display area 705 of the cloud scan screen 701. Then, the series of processes shown in FIG. 8 ends.
On the other hand, if the MFP server communication unit 310 determines in S802 that the transmission of the data corresponding to the result of reading the document to the MFP server 120 has been successful, the process proceeds to S803.

In S803, the MFP communication unit 406 receives the processing request sent from the MFP 100, and registers the processing content corresponding to the processing request in the job control unit 403. Based on the information registered as an attribute in the processing request, the job control unit 403 determines whether a cloud storage to which data corresponding to the document reading results is to be sent has been specified, or whether URL information has been associated with the data.

If the job control unit 403 determines in S803 that the association of URL information with the data corresponding to the document reading result has not been specified and that a cloud storage destination has been specified, the process proceeds to S823. In S823, the job control unit 403 transmits the data corresponding to the document reading result (in other words, the data of the scanned image) to the cloud storage specified as the destination via the cloud storage communication unit 405.
In this case, since no URL information is associated with the data, no file on the cloud storage to be reflected by the data is specified. Therefore, the data may be placed in, for example, a directory set as a default in the cloud storage to which the data is to be sent.
The file name of the data may also be determined based on conditions set in the cloud storage, for example. As a specific example, the file name of the target data may be determined based on information regarding the date and time when the scan was performed.
In addition, the file format of the data may be determined based on conditions set in the cloud storage, for example. As a specific example, the target data may be stored as a PDF file.
Of course, the above is merely an example and may be modified as appropriate.

On the other hand, if the job control unit 403 determines in step S803 that the association of URL information with data corresponding to the document reading result has been specified, the process proceeds to step S804.
In this embodiment, for convenience, the selection state of the cloud storage in the display area 702 on the cloud scan screen 701 and the input state in the check box 703 are mutually exclusive controlled by the operation control unit 301. Specifically, when the check box 703 is checked, the display area 702 is controlled to recognize that no cloud storage has been selected. In other words, the input state in the check box 703 takes priority over the selection state of the cloud storage in the display area 702.

In S804, the job control unit 403 causes the image processing unit 404 to analyze the target data, and based on the results of the analysis, attempts to identify a URL for accessing a file (electronic document) on cloud storage corresponding to the data.
For example, the image processing unit 404 may recognize URL information that is assigned in a predetermined description format (e.g., a two-dimensional barcode format) to the cover page added to the data during cloud printing, and identify the URL based on the URL information.
As a specific example, image analysis may be performed on a partial image corresponding to a specific area of a scanned image obtained by reading a document (printed material) to recognize URL information presented in the partial image.
The scanned image resulting from the reading of the document corresponds to an example of a “first image.” The process of S804 corresponds to an example of a “first specific process.”

When the URL is specified, the job control unit 403 accesses the cloud storage corresponding to the URL via the cloud storage communication unit 405 and attempts to obtain a file (electronic document) corresponding to the data from the cloud storage. At this time, the job control unit 403 may use, for example, account information corresponding to the target cloud storage stored in a predetermined storage area by the data storage unit 402 for authentication to access the cloud storage.
Then, the job control unit 403 determines whether or not a file on the cloud storage corresponding to the target data has been acquired.

If the job control unit 403 determines in S804 that it has been unable to obtain a file on cloud storage that corresponds to the target data, it advances the process to S821. As a specific example, if access to cloud storage fails, if authentication fails, or if the corresponding file does not exist on cloud storage, then obtaining the target file in S804 fails. In this case, the job control unit 403 has been unable to obtain the file that is the target of the process relating to reflecting the proofreading results shown on the manuscript based on the data corresponding to the results of reading the manuscript, which will be described in detail later as the process of S805 to S818.

In S821, the job control unit 403 instructs the data storage unit 402 to store a history of the job processing in a specified storage area. In this case, information indicating a failure in the cloud scan processing is stored as the history of the job processing. Thereafter, the series of processes shown in FIG. 8 ends.

On the other hand, if the job control unit 403 determines in S804 that a file on the cloud storage corresponding to the target data has been acquired, the process proceeds to S805. In S805, the job control unit 403 requests the image processing unit 404 to create a rendering image of the file (telegram document) acquired in S804. In response to this request, the image processing unit 404 generates a rendering image by performing rendering on the file. Note that, for convenience, in this embodiment, the file is assumed to be an electronic document in text format or an electronic document described based on a predetermined markup language. Also, for convenience, the markup language is assumed to be XML (Extensible Markup Language), but this does not necessarily limit the available markup language, and other markup languages may be used. Also, the data of the rendering image is assumed to be data in bitmap format.
The rendering image corresponds to an example of a "second image."

In S806, the image processing unit 404 compares the scanned image corresponding to the result of reading the document in S801 with the rendering image created in S805, and obtains a difference image corresponding to the result of the comparison.
Note that the scanned image and the rendered image may differ in size and resolution. In such a case, for example, the image processing unit 404 may adjust the size and resolution of at least one of the scanned image and the rendered image. As a specific example, when creating a rendered image, the image processing unit 404 may control the size and resolution of the rendered image to be created in accordance with the size and resolution of the scanned image.
In addition, the image processing unit 404 may adjust the inclination of at least one of the scanned image and the rendered image, assuming a situation in which the scanned image is inclined relative to the rendered image. As a specific example, the image processing unit 404 may extract a difference between the scanned image and the rendered image while rotating the rendered image, and specify the rotation angle at which the difference is minimum as the rotation angle of the rendered image when comparing the scanned image and the rendered image.

In S807, the image processing unit 404 recognizes the difference image acquired in S806 as an image showing writing made to the printout of the electronic document from which the rendering image is generated (hereinafter also referred to as a "writing image"). The image processing unit 404 breaks down the writing image into proofreading symbols and natural language written as a result of proofreading the printout of the electronic document.
Examples of proofreading symbols include symbols defined by the International Organization for Standardization as ISO 5776 and symbols defined by the Japanese Industrial Standards as JIS Z 8208.
The method of decomposing the written image into the review symbol and the natural language is not particularly limited. As a specific example, the written image may be decomposed into the review symbol and the natural language by pattern matching using a recognizer constructed by learning the characteristics of the review symbol and the natural language based on so-called machine learning.
The image processing unit 404 also performs various analytical processes (e.g., image analysis, etc.) on the review symbols and natural language extracted from the written image by the above-mentioned decomposition process, thereby recognizing correction instructions indicated by the review symbols and converting the natural language into text information. The conversion of natural language into text information can be realized, for example, by OCR processing.
Note that the natural language written on the printed material during proofreading is used, for example, in combination with proofreading symbols, and plays a supplementary role in presenting correction candidates, etc.
The processes in S806 and S807 correspond to an example of a "second identification process."

In S808, the image processing unit 404 identifies the coordinates of the proofreading symbol on the rendered image. In this disclosure, the coordinates on the rendered image refer to an area surrounded by four points represented by the X-axis and Y-axis on a bitmap described as a two-dimensional array. Since the above-mentioned written image (difference image) is obtained by comparing the scanned image and the rendered image in S806, it goes without saying that it is possible to identify the positional correspondence between the rendered image and the written image.

In S809, the image processing unit 404 extracts an area corresponding to the coordinates identified in S808 from the rendering image. As described above, the rendering image is an image generated by rendering the electronic document to be reflected (in other words, the electronic document corresponding to the printed matter on which notes have been added during proofreading). Therefore, it is expected that the extracted area will present the character string that the user targeted for correction during proofreading.

In S810, the image processing unit 404 identifies the character string presented in the area on the rendering image extracted in S809. Note that the data of the area extracted in S809 is image data in a format such as a bitmap. Therefore, the image processing unit 404 may identify the character string presented in the area by, for example, performing OCR processing on the area (i.e., the image in a format such as a bitmap). The image processing unit 404 may convert the character string identified from the area into text information and store it.

In S811, the image processing unit 404 searches the character string identified in S810 (i.e., the character string presented in the area on the rendering image) from the electronic document to be reflected. As a specific example, the image processing unit 404 may search the character string converted into text information in S810 from the electronic document to be reflected based on a character string comparison with text information indicating the contents of the electronic document to be reflected.

In S812, the image processing unit 404 identifies at least some of the candidates for character strings in the electronic document searched for in S811 as reflection candidates. Then, the image processing unit 404 processes the electronic document to control the display mode so that the character strings that are reflection candidates are displayed in a different mode from other character strings.
In this embodiment, the image processing unit 404 processes the electronic document to be processed so that the character strings that are candidates for reflection are highlighted.

The processing for highlighting the character strings that are candidates for reflection may be appropriately changed according to the format of the electronic document to be processed.
As a specific example, when an electronic document written in a specified markup language is the target, a process for assigning highlight attributes to the character strings that are candidates for reflection based on the format of the markup language may be performed on the electronic document.
As another example, when a text-format electronic document is the target, the appearance position of the character string that is the reflection candidate in the electronic document may be specified, and information about the appearance position may be stored in a predetermined storage area. Note that, for example, information indicating the appearance position in a text file may be information indicating the number of characters from the beginning of the file. Note that, the information indicating the appearance position is used to specify the position to be highlighted when rendering the electronic document.

As explained above, by carrying out processing related to control of the display mode, such as highlighting, it becomes possible to identify candidates for the character string to be corrected, for example, when a similar character string appears multiple times in the electronic document to be reflected. Details of this processing will be described separately later.

In step S813, the image processing unit 404 generates a rendering image by performing rendering on the electronic document that has been processed in step S812 to highlight the character strings that are candidates for reflection.
For example, when an electronic document written in a specified markup language is targeted, a rendering process is performed in the same manner as in normal cases, and the area corresponding to the character string that is a candidate for reflection is highlighted.
As another example, when a text-format electronic document is the target, a process for performing highlight processing is executed at the time of rendering. Specifically, when generating a rendering image, a process for changing the background color of an area corresponding to a character string that is a candidate for reflection is executed based on information regarding the appearance position of the character string.
Note that a rendering image generated by performing rendering on an electronic document that has been subjected to processing related to control of the display mode, such as highlight processing, as exemplified above, corresponds to an example of a "third image."

In S814, the image processing unit 404 compares the rendering image generated in S805 with the rendering image generated in S813. Then, based on the result of the comparison, the image processing unit 404 identifies coordinates on the rendering image generated in S805 that correspond to the area in the rendering image generated in S813 that has been subjected to highlight processing.

In S815, the image processing unit 404 determines whether the coordinates of the highlighted area identified in S814 match the coordinates of the revision symbol identified in S808. Note that the size of the highlighted area does not necessarily match the size of the revision symbol area. Therefore, for example, the image processing unit 404 may determine that the coordinates of these areas match when the size of a partial area that overlaps with the other area in at least one of these areas is equal to or greater than a predetermined percentage (e.g., equal to or greater than half).

If the image processing unit 404 determines in S815 that the coordinates of the highlighted area do not match the coordinates of the proofreading symbol, the process proceeds to S812. In this case, the image processing unit 404 identifies another candidate and executes the process from S812 onward for the other candidate.

On the other hand, if the image processing unit 404 determines in S815 that the coordinates of the highlighted area match the coordinates of the revision mark, the process proceeds to S816.
The process of identifying the reflection candidates shown as S808 to S815 corresponds to an example of the "third identification process."

In S816, the job control unit 403 determines whether the specified operation mode is the review mode based on the input state of the check box 706 registered as the processing content of the cloud scan for the job. Specifically, the job control unit 403 determines that the operation mode is the review mode when the check box 706 is checked.

If the job control unit 403 determines in S816 that the operating mode specified is not the review mode, the process proceeds to S822. In S822, the image processing unit 404 corrects the character string in the electronic document identified as a candidate for reflection in S812 according to the content indicated by the review symbol. As a specific example, if the review symbol is a strikethrough, the image processing unit 404 deletes the character string that is a candidate for reflection. As another example, if the review symbol indicates a correction to a different character string, the image processing unit 404 replaces the character string that is a candidate for reflection with the different character string.

On the other hand, if the job control unit 403 determines that the operation mode specified in S816 is the review mode, the process proceeds to S817. In S817, the image processing unit 404 adds a description according to the correction content indicated by the review symbol to the character string in the electronic document identified as a reflection candidate in S812 as review information. For example, in the case of an XML format electronic document, the image processing unit 404 may add and reflect information indicating the above-mentioned correction content to the electronic document using an XML tag indicating that it is review information. As a more specific example, if the review symbol is a strikethrough, the image processing unit 404 adds an XML tag indicating deletion to the reflection candidate character string as review information. As another example, if the review symbol means a correction to another character string, the image processing unit 404 adds an XML tag indicating a correction to the reflection candidate character string as review information. Furthermore, if the electronic document to be reflected is in the file format of a specific document editing application, the proofreading information (correction contents) may be reflected in a format that conforms to the proofreading function of that document editing application. For example, if the specific document editing application is Microsoft's Word (trademark), the correction contents corresponding to the proofreading symbols detected from the scanned image may be added to the electronic document as a "record of change history (correction history)" in the proofreading function of that application.

In S818, the job control unit 403 transmits the electronic document modified in S817 or S822 to the cloud storage via the cloud storage communication unit 405, and instructs the cloud storage to update the electronic document to be reflected. As a result, the electronic document to be reflected that is stored in the cloud storage is overwritten with the electronic document transmitted from the job control unit 403 (i.e., the electronic document modified in S817 or S822).
In addition, if the cloud storage manages versions of the electronic document to be processed, the electronic document modified in the overwrite process is treated as a new version of the electronic document to be reflected. As a specific example, when the process of S817 is executed, the new version of the electronic document to be reflected in S817 is added to the electronic document to be reflected that is stored in the cloud storage. By performing such management, it is possible to control, for example, to return the electronic document stored in the cloud storage to the version before the overwrite process, or to present the modified part of the new version as a difference.

In S819, the job control unit 403 causes the data storage unit 402 to store a history of the series of job processing described above in a predetermined storage area. The information stored as the history includes, for example, the file name of the electronic document to be reflected, the processing date and time, and information regarding the success or failure of the processing. In this case, information indicating the success of the cloud scan processing is recorded as the history of the job processing.

By applying the above-mentioned processing, even if the proofreading results are handwritten on a printed copy of an electronic document, it is possible to automatically reflect corrections corresponding to the contents indicated by the proofreading results in the electronic document stored in cloud storage. Furthermore, according to the information processing system of this embodiment, it is also possible to reflect, as proofreading information, corrections corresponding to the contents indicated by the proofreading results in the electronic document stored in cloud storage.

<Modification>
A modified example of the information processing system according to the present embodiment will be described below. In this modified example, when a correction based on the contents indicated by the proofreading results is reflected as proofreading information in an electronic document, a clip image indicating the contents of the proofreading that were added by hand or the like to the printed matter during the proofreading as a supplement is associated with the electronic document.
In addition, in this modified example, the description will focus on the parts that are different from the previously described embodiment, and detailed descriptions of parts that are substantially the same as the previously described embodiment will be omitted.

Figure 9 is a flowchart showing an example of processing of the information processing system according to this modified example, and shows an example of processing related to realizing the cloud scan function. The series of processing shown in Figure 9 is started, for example, by pressing button 704 on the cloud scan screen 701 shown in Figure 7.

In step S901, the operation control unit 301 registers the processing content of the cloud scan in the job control unit 303 according to the selection state of the cloud storage in the display area 702 of the cloud scan screen 701 and the input state of the check boxes 703 and 706.
The job control unit 303 requests the reading processing unit 306 to execute processing related to reading the document placed on the scanner unit 104 in accordance with the registered processing content. In response to this request, the reading processing unit 306 controls the operation of the scanner unit 104 to execute processing related to reading the document placed on the reading surface of the scanner unit 104.

In S902, the MFP server communication unit 310 transmits data corresponding to the result of reading the document in S901 (i.e., scanned image data) as a processing request to the MFP server 120. At this time, information regarding the cloud storage selected in the display area 702 and the input states of the check boxes 703 and 706 are also notified to the MFP server 120.
Then, the MFP server communication unit 310 determines whether or not the transmission of the data corresponding to the result of reading the document to the MFP server 120 has been successful.
If the MFP server communication unit 310 determines in S902 that the transmission of data corresponding to the document reading result to the MFP server 120 has failed, the process proceeds to S922. In S922, the operation control unit 301 of the MFP 100 displays information about the error that has occurred in the display area 705 of the cloud scan screen 701. Then, the series of processes shown in FIG. 9 ends.
On the other hand, if the MFP server communication unit 310 determines in S902 that the transmission of the data corresponding to the result of reading the document to the MFP server 120 has been successful, the process proceeds to S903.

In S903, the MFP communication unit 406 receives the processing request sent from the MFP 100, and registers the processing content corresponding to the processing request in the job control unit 403. Based on the information registered as an attribute in the processing request, the job control unit 403 determines whether a cloud storage device has been specified as a destination for data corresponding to the results of scanning the original document, or whether URL information has been associated with the data.

If the job control unit 403 determines in S903 that no URL information has been associated with the data corresponding to the results of scanning the original document, and that a cloud storage destination has been specified, the process proceeds to S925. In S925, the job control unit 403 transmits the data corresponding to the results of scanning the original document (in other words, the data of the scanned image) to the cloud storage specified as the destination, via the cloud storage communication unit 405.

On the other hand, if the job control unit 403 determines in S903 that an association of URL information with data corresponding to the document reading results has been specified, the process proceeds to S904.

In S904, the job control unit 403 causes the image processing unit 404 to analyze the target data, and based on the results of the analysis, attempts to identify a URL for accessing a file (electronic document) on cloud storage corresponding to the data.
Then, the job control unit 403 determines whether or not a file on the cloud storage corresponding to the target data has been acquired.

If the job control unit 403 determines in S904 that a file on the cloud storage corresponding to the target data could not be acquired, the process proceeds to S923. As a specific example, if access to the cloud storage fails, if authentication fails, or if the corresponding file does not exist on the cloud storage, acquisition of the target file fails in S904.

In S923, the job control unit 403 instructs the data storage unit 402 to store a history of the job processing in a specified storage area. In this case, information indicating a failure in the cloud scan processing is recorded as the history of the job processing. Thereafter, the series of processes shown in FIG. 9 ends.

On the other hand, if the job control unit 403 determines in S904 that a file on the cloud storage corresponding to the target data has been acquired, the process proceeds to S905. In S905, the job control unit 403 requests the image processing unit 404 to create a rendering image of the file (telegram document) acquired in S904. In response to this request, the image processing unit 404 generates a rendering image by rendering the file.

In S906, the image processing unit 404 compares the scanned image resulting from reading the original in S901 with the rendering image created in S905, and obtains a difference image resulting from the comparison.

In S907, the image processing unit 404 recognizes the difference image acquired in S906 as a writing image indicating writing made to the printout of the electronic document from which the rendering image is generated. The image processing unit 404 performs a predetermined image processing on the writing image to decompose the writing image into a proofreading symbol and natural language. The image processing unit 404 also recognizes the content of the correction instruction represented by the proofreading symbol decomposed from the writing image, and converts the natural language decomposed from the writing image into text information. The natural language can be converted into text information, for example, by OCR processing. Note that the natural language writing is used, for example, in combination with the proofreading symbol and plays a supplementary role in presenting correction candidates, etc.

In S908, the image processing unit 404 identifies the coordinates of the revision symbol on the rendering image.

In S909, the image processing unit 404 extracts the area corresponding to the coordinates identified in S908 from the rendering image.

In S910, the image processing unit 404 identifies the character string presented in the area on the rendering image extracted in S909.

In S911, the image processing unit 404 searches the electronic document to be reflected for the character string identified in S910.

In S912, the image processing unit 404 performs processing on the target electronic document to control the display manner so that each of a series of candidate character strings in the electronic document searched for in S911 is displayed in a manner different from other character strings.
In this modified example, similarly to the embodiment described above, the image processing unit 404 processes the target electronic document so that the character strings that are candidates for reflection are highlighted.

In S913, the image processing unit 404 generates a rendering image by performing rendering on the electronic document that has been processed to highlight the character strings corresponding to each of the series of candidates searched for in S912.

In S914, the image processing unit 404 compares the rendering image generated in S905 with the rendering image generated in S913. Then, based on the result of the comparison, the image processing unit 404 identifies coordinates on the rendering image generated in S905 that correspond to the area in the rendering image generated in S913 that has been subjected to highlight processing.

In S915, the image processing unit 404 determines whether the coordinates of the highlighted area identified in S914 match the coordinates of the revision symbol identified in S908. Note that the size of the highlighted area does not necessarily match the size of the revision symbol area. Therefore, for example, the image processing unit 404 may determine that the coordinates of these areas match when the size of a partial area that overlaps with the other area in at least one of these areas is equal to or greater than a predetermined percentage (e.g., equal to or greater than half).

If the image processing unit 404 determines in S915 that the coordinates of the highlighted area do not match the coordinates of the proofreading symbol, the process proceeds to S912. In this case, the image processing unit 404 identifies another candidate and executes the process from S912 onward for the other candidate.

On the other hand, if the image processing unit 404 determines in S915 that the coordinates of the highlighted area match the coordinates of the revision mark, it advances the process to S916.

In S916, the job control unit 403 determines whether the specified operation mode is the review mode based on the input state of the check box 706 registered as the cloud scan processing content for the job.

If the job control unit 403 determines in S916 that the specified operating mode is not the review mode, the process proceeds to S924. In S924, the image processing unit 404 corrects the character string in the electronic document identified in S912 as a candidate for reflection according to the content indicated by the review symbol.

On the other hand, if the job control unit 403 determines in S916 that the operation mode specified is the proofreading mode, the process proceeds to S917. In S917, the image processing unit 404 determines whether or not the recognition of the content of the correction instruction represented by the proofreading symbol resolved from the written image in S907 has been successful.
As a specific example, the image processing unit 404 may determine that it has failed to recognize the contents of the correction instruction if, even after applying image processing such as pattern matching to a partial image of an area in the written image that is recognized as a proofreading symbol, it is still difficult to identify the corresponding proofreading symbol.
Furthermore, the image processing unit 404 may determine that the recognition of the contents of the correction instruction has failed if it is difficult to identify the corresponding character string even after performing image processing such as OCR processing on a partial image of an area in the written image that has been recognized as natural language. Note that, if the image processing unit 404 succeeds in identifying the character string, it may treat the recognition of the contents of the correction instruction as successful even if the character string has been erroneously converted into a character string different from the actual one.

If the image processing unit 404 determines in S917 that it has succeeded in recognizing the content of the correction instruction represented by the proofreading symbol resolved from the written image, the process proceeds to S918. In S918, the image processing unit 404 adds correction information corresponding to the content indicated by the proofreading symbol to the character string in the electronic document identified as a reflection candidate in S912, as proofreading information in the electronic document.
Then, in S919, the image processing unit 404 associates the partial image of the portion of the written image (difference image) acquired in S907 that corresponds to the instruction related to the proofreading with the proofreading information reflected in the electronic document in S918. As a specific example, the image processing unit 404 may extract a partial image of the region where there is writing around the proofreading symbol from the written image, and associate the partial image with the proofreading information.

On the other hand, if the image processing unit 404 determines in step S917 that it has failed to recognize the content of the correction instruction represented by the proofreading symbol separated from the written image, it advances the process to step S919.
In this case, in S919, the image processing unit 404 generates proofreading information associated with a partial image of the part of the written image that corresponds to the proofreading instructions, and reflects the proofreading information on the character string in the electronic document identified as a candidate for reflection in S912.

In S920, the job control unit 403 transmits the electronic document modified in S918 and S919, S919, or S924 to the cloud storage via the cloud storage communication unit 405. Then, the job control unit 403 instructs the cloud storage to update the electronic document to be reflected. As a result, the electronic document to be reflected that is stored in the cloud storage is overwritten with the electronic document transmitted from the job control unit 403 (i.e., the electronic document modified in S918 and S919, S919, or S924).

In S921, the job control unit 403 causes the data storage unit 402 to store a history relating to the processing of the above-mentioned series of jobs in a specified storage area. The information stored as the history includes, for example, the file name of the electronic document to be reflected, the processing date and time, and information relating to the success or failure of the processing. In this case, information indicating the success of the cloud scan processing is recorded as the history relating to the job processing.

By applying the above-mentioned processing, even if the proofreading results are handwritten on a printout of an electronic document, it is possible to automatically reflect the corrections corresponding to the contents indicated by the proofreading results on the electronic document stored in the cloud storage. Furthermore, the information processing system according to the present embodiment also makes it possible to reflect, as proofreading information, the corrections corresponding to the contents indicated by the proofreading results on the electronic document stored in the cloud storage.
Furthermore, in the information processing system according to this modified example, a clip image showing instructions corresponding to the proofreading results, which corresponds to the results of reading a printout on which instructions corresponding to the proofreading results have been written by hand or the like, is associated with the proofreading information. Therefore, even if an error occurs in the recognition of the proofreading contents written on the printout by image processing such as OCR processing, the user can determine whether or not to reflect the corrections while checking the instructions corresponding to the proofreading results shown in the clip image.

<Other embodiments>
The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a recording medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

100 MFP
120 MFP server 121 Cloud storage 403 Job control unit 404 Image processing unit

Claims (9)

a scanner unit that reads a printed matter to which information indicating the location on cloud storage where the original electronic document is stored is attached as an image to generate a first image, and an operation unit that displays a screen for receiving from the user a first instruction specifying whether or not to reflect correction instructions written on the printed matter in the original electronic document stored in the location indicated by the information , a second instruction specifying a cloud storage to which the first image is to be sent from a list of cloud storages available to the user, and a third instruction specifying whether or not to add the correction instructions in a format conforming to a proofreading function of a document editing application capable of editing the original electronic document;
a transmission means for transmitting the first image received by the receiving means to the cloud storage specified in the second instruction when the first instruction received by the receiving means specifies that the correction instruction should not be reflected in the original electronic document and the second instruction specifies a cloud storage to which the first image is to be transmitted;
a generating means for generating a second image by performing rendering on the original electronic document when the first instruction received by the receiving means specifies that the correction instruction should be reflected in the original electronic document, and when the original electronic document can be obtained from a location on the cloud storage based on information indicating a location on the cloud storage where the original electronic document is stored, the information being obtained by analyzing the first image received by the receiving means, and generating a difference image by extracting a difference between the first image received by the receiving means and the generated second image;
a specifying means for performing an analysis process and an OCR process on the difference image generated by the generating means, thereby recognizing the correction instruction indicated by the proofreading symbol included in the difference image and text information in natural language included in the difference image, and further specifying a character string in the electronic document that is a target for reflecting the correction instruction based on the coordinates of the proofreading symbol;
when the first instruction received by the receiving means specifies that the correction instruction should be reflected in the original electronic document, and the original electronic document can be obtained from the location on the cloud storage based on the information indicating the location on the cloud storage where the original electronic document is stored, which is obtained by analyzing the first image received by the receiving means, and when the third instruction specifies that the correction instruction should be added in a format conforming to a proofreading function of the document editing application, a process is executed to add the correction content based on the correction instruction to a character string in the electronic document identified by the identifying means as a record of a change history in the proofreading function of the document editing application,
an execution means for executing a process of reflecting the correction instruction in the original electronic document specified by the specification means, when the first instruction received by the receiving means specifies that the correction instruction should be reflected in the original electronic document, and the original electronic document can be acquired from the location on the cloud storage based on the information indicating the location on the cloud storage where the original electronic document is stored, which is obtained by analyzing the first image received by the receiving means, and when the third instruction specifies that the correction instruction should not be added in a format conforming to a proofreading function of the document editing application, the correction content based on the correction instruction is not recorded in a change history in the proofreading function of the document editing application, but is reflected in a character string in the electronic document specified by the specification means;
An information processing device comprising: The information processing device according to claim 1, further comprising a storage means for storing information indicating that reflecting the correction instructions in the original electronic document has failed when the first instruction received by the receiving means specifies that the correction instructions should be reflected in the original electronic document, and when the original electronic document cannot be obtained from the location on the cloud storage where the original electronic document is stored based on the information indicating the location on the cloud storage where the original electronic document is stored, which is obtained by analyzing the first image received by the receiving means. 2. The information processing device according to claim 1, wherein the execution means executes a process in which the first instruction received by the receiving means specifies that the correction instruction should be reflected in the original electronic document, and the execution means is capable of retrieving the original electronic document from a location on the cloud storage based on the information indicating a location on the cloud storage where the original electronic document is stored, obtained by analyzing the first image received by the receiving means, and the execution means executes a process in which, when the third instruction specifies that the correction instruction should be added in a format conforming to a proofreading function of the document editing application, and the correction content based on the correction instruction is deletion, a description indicating the deletion of a character string in the electronic document identified by the identification means is added to the electronic document as a record of a change history in the proofreading function of the document editing application. 2. The information processing device according to claim 1, wherein the execution means executes a process in which the first instruction received by the receiving means specifies that the correction instruction should be reflected in the original electronic document, and the execution means is capable of retrieving the original electronic document from a location on the cloud storage based on the information indicating a location on the cloud storage where the original electronic document is stored, obtained by analyzing the first image received by the receiving means, and the execution means executes a process in which, when the third instruction specifies that the correction instruction should be added in a format conforming to a proofreading function of the document editing application, and further when the correction content based on the correction instruction is a correction to another character string, a description indicating that the character string in the electronic document identified by the identification means is to be corrected to the other character string is executed to the electronic document as a record of a change history in the proofreading function of the document editing application. 5. The information processing device according to claim 1, wherein, when the first instruction received by the receiving means specifies that the correction instructions should be reflected in the original electronic document, and the execution means is capable of retrieving the original electronic document from a location on the cloud storage based on the information indicating the location on the cloud storage where the original electronic document is stored, which is obtained by analyzing the first image received by the receiving means, and the execution means specifies that the correction instructions should be added in a format conforming to the proofreading function of the document editing application, further extracts a partial image of a portion corresponding to the correction instructions from the difference image and adds the extracted partial image to the electronic document as a record of change history in the proofreading function of the document editing application. The information processing device according to claim 1, characterized in that the electronic document is an XML format electronic document. The information processing device according to claim 1, characterized in that the location where the electronic document from which the printed matter is printed is stored is specified by a URL or a directory path written in the first image in the form of a character string or a barcode. An information processing method executed by an information processing device,
a receiving step of receiving the first image, the first instruction, the second instruction, and the third instruction from a device including: a scanner unit that reads a printed matter to which information indicating the location on the cloud storage where the original electronic document is stored is attached as an image to generate a first image; and an operation unit that displays a screen for receiving from the user a first instruction specifying whether or not to reflect correction instructions written on the printed matter in the original electronic document stored in the location indicated by the information, a second instruction specifying a cloud storage to which the first image is to be sent from a list of cloud storages available to the user, and a third instruction specifying whether or not to add the correction instructions in a format conforming to a proofreading function of a document editing application capable of editing the original electronic document;
a transmission step of transmitting the first image received in the receiving step to the cloud storage specified in the second instruction when the first instruction received in the receiving step specifies that the modification instruction should not be reflected in the original electronic document and the second instruction specifies a cloud storage to which the first image is to be transmitted;
a generating step of generating a second image by performing rendering on the electronic document to be printed, if the first instruction received in the receiving step specifies that the correction instruction should be reflected in the electronic document to be printed, and if the electronic document to be printed can be obtained from a location on the cloud storage based on the information indicating a location on the cloud storage where the electronic document to be printed is stored, obtained by analyzing the first image received in the receiving step, and generating a difference image by extracting a difference between the first image received in the receiving step and the generated second image;
a step of performing an analysis process and an OCR process on the difference image generated in the generation step, thereby recognizing the correction instruction indicated by the proofreading symbol included in the difference image and text information in natural language included in the difference image, and further, identifying a character string in the electronic document that is a target for reflecting the correction instruction based on the coordinates of the proofreading symbol;
if the first instruction received in the receiving step specifies that the correction instruction should be reflected in the original electronic document, and the original electronic document can be obtained from the location on the cloud storage based on the information indicating the location on the cloud storage where the original electronic document is stored, which is obtained by analyzing the first image received in the receiving step, and the third instruction specifies that the correction instruction should be added in a format conforming to a proofreading function of the document editing application, a process is executed to add the correction content based on the correction instruction to a character string in the electronic document identified in the identifying step as a record of a change history in the proofreading function of the document editing application,
an execution step of executing a process of reflecting the correction instruction in the original electronic document identified in the identification step, when the first instruction received in the receiving step specifies that the correction instruction should be reflected in the original electronic document, and the original electronic document can be acquired from the location on the cloud storage based on the information indicating the location on the cloud storage where the original electronic document is stored, which is obtained by analyzing the first image received in the receiving step, and the third instruction specifies that the correction instruction should not be added in a format conforming to a proofreading function of the document editing application;
13. An information processing method comprising: A program for causing a computer to function as each of the means of an information processing device according to any one of claims 1 to 7.

Images