JP7558730B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to information processing related to financial statements.

Conventionally, optical character recognition (OCR) technology is known for reading character information from image data.

JP 2004-246575 A (Published on September 2, 2004)

However, in the conventional technology described above, when OCR processing is performed on financial statements that are particularly unclear, there is a problem that input accuracy decreases and data different from the intended account items or figures may be set. On the other hand, when financial statement data is manually entered to improve input accuracy, there is a problem that it takes time and effort from the user.

One aspect of the present invention has been made in consideration of the above problems, and aims to realize an information processing device that improves user convenience and input accuracy when inputting financial statement data.

In order to solve the above problem, an information processing device according to one embodiment of the present invention includes an acquisition unit that acquires an image of a financial statement, a preprocessing unit that performs a predetermined preprocessing on the image acquired by the acquisition unit according to the type of the financial statement, a reading unit that reads text data included in the image preprocessed by the preprocessing unit and assigns to each piece of text data a numerical value or a preregistered account item or a category name in the financial statement based on the degree of approximation, and a presentation unit that presents each piece of text data whose degree of approximation is less than a predetermined degree so that the user can modify it.

According to one aspect of the present invention, it is possible to realize an information processing device that improves user convenience and input accuracy when inputting financial statement data.

1 is a functional block diagram of an information processing system according to a first embodiment. 1 is a schematic diagram of an information processing system according to a first embodiment. 4 is a flowchart showing a process flow according to the first embodiment. FIG. 4 is a diagram illustrating an example of pre-processing according to the first embodiment. 13 is an example of a diagram illustrating a portion of a financial statement in which text data has been rearranged by the preprocessing unit. 13 is a conceptual diagram showing how the reading unit compares the text data of the financial statement with the account items registered in the storage unit for each category. FIG. FIG. 13 is a diagram showing an example of a correction screen displayed on a terminal device based on information transmitted from an information processing device. 10 is a flowchart showing a process flow according to a second embodiment. FIG. 11 is a diagram illustrating an example of pre-processing according to the second embodiment.

[Embodiment 1]
Hereinafter, one embodiment of the present invention will be described in detail.

[1. Example of information processing system configuration]
The information processing system according to this embodiment is a system in which an information processing device performs a predetermined preprocessing on an image of a financial statement (financial statement) transmitted from a terminal device, reads text data in the image, and performs processing related to credit management, for example. In this specification, text data refers to data showing character strings such as numbers and account headings, or the character strings themselves.

FIG. 1 is a functional block diagram of an information processing system 1 according to this embodiment. The information processing system 1 includes an information processing device 10 and a terminal device 30. The information processing device 10 and the terminal device 30 are connected to each other via a network so that they can communicate with each other.

The information processing device 10 is a device that performs image analysis of financial statements, and includes a control unit 12, a memory unit 20, and a communication unit 22.

The control unit 12 is a CPU or a control device equivalent thereto that controls the entire information processing device 10, and also functions as an acquisition unit 14, a preprocessing unit 15, a reading unit 16, a presentation unit 17, and a calculation unit 18.

The acquisition unit 14 acquires images of financial statements. The source from which the acquisition unit 14 acquires images of financial statements is not limited, and may be configured to acquire images uploaded onto a web server from the terminal device 30, for example.

The preprocessing unit 15 performs a predetermined preprocessing on the image acquired by the acquisition unit 14 according to the type of financial statement. An example will be described later, but this preprocessing makes it possible to improve the accuracy when the reading unit 16 reads text data. Furthermore, when the preprocessing requires the reading of a financial statement, the preprocessing unit 15 performs processing via the reading unit 16.

In the examples of this specification, the types of financial statements are described as being either a balance sheet, a profit and loss statement, or a selling and administrative expenses (statement), but the types are not limited to these and other financial statements may also be processed.

The reading unit 16 has a mechanism for performing OCR processing, and reads text data contained in the image preprocessed by the preprocessing unit 15, and assigns to each piece of text data either a numerical value or an account item or a category name in a financial statement that has been registered in advance in the storage unit 20, etc., based on the degree of similarity. The reading unit 16 also performs processing to detect text data and predetermined objects such as straight lines within the image.

The processing by the reading unit 16 may be configured to use a machine learning method such as Faster R-CNN (Regions with Convolutional Neural Networks). In this configuration, the storage unit 20 stores functions and parameters that define the learning model referenced by the reading unit 16.

The presentation unit 17 presents each piece of text data, etc., that is less than a predetermined degree of similarity to either a numerical value, or an account item or a category name in a financial statement that has been registered in advance in the storage unit 20, etc., in a manner that allows the user to modify it. The presentation unit 17 also performs a process of presenting each piece of information, which will be described later, to the user.

The calculation unit 18 calculates the judgment result regarding credit management by referring to the account items assigned to each text data read by the reading unit 16 and the numerical values (amounts) assigned to the account items. Here, the judgment result regarding credit management is, for example, rating information indicating the creditworthiness of the financial situation of the company related to the target financial statement.

The memory unit 20 is a storage device that stores various information, such as a database showing a list of account items in each financial statement.

Furthermore, account items are registered in the storage unit 20 by type of financial statement or by category in the financial statement. Here, a financial statement category is a classification that broadly classifies the types of account items, and in the case of a balance sheet, for example, it means items such as "current assets," "fixed assets," "current liabilities," and "fixed liabilities" in each section such as the assets section and the liabilities section. However, each section such as the assets section described above may also be included in the financial statement category.

The communication unit 22 performs communication processing with external devices such as the terminal device 30.

The terminal device 30 is a device such as a personal computer, smartphone, or tablet, and is equipped with a display unit 32 and a communication unit 34, etc.

The display unit 32 is a display that displays moving images, text, etc. The display unit 32 may also be realized as a touch panel, allowing input operations to be performed via the display unit 32.

In addition, in this embodiment, the process of presenting text data, etc. to the user by the presentation unit 17 is performed by displaying related information on the display unit 32. In other words, the presentation unit 17 performs the process of presenting text data, etc. to the user by transmitting the data displayed on the display unit 32 to the terminal device 30 via the communication unit 22.

The communication unit 34 performs communication processing with external devices such as the information processing device 10. For example, the communication unit 34 transmits image data showing a financial statement, which is at least temporarily held by the terminal device 30, to the information processing device 10 based on a user operation.

In addition, as an alternative to the configuration shown in FIG. 1, the information processing device 10 may be configured to be realized as, for example, a web server that provides a website that serves as an interface, a cloud server, and an API (Application Programming Interface) server. In other words, the information processing device 10 may be configured by a web server, a cloud server, and an API server.

Figure 2 is a schematic diagram showing this configuration. In the example of Figure 2, the web server acquires an image of the financial statement uploaded to its own device, and transmits information showing the acquired image of the financial statement to the cloud server. The cloud server also executes the processes corresponding to the preprocessing unit 15 and the reading unit 16 by appropriately calling the above API, and transmits the read character information to the web server. Next, based on the information received from the cloud server, the web server transmits information showing a screen for correcting the text data in the financial statement to the terminal device 30 that uploaded the image of the financial statement to its own device.

2. Examples of processing by the information processing system
The flow of processing in the information processing system 1 according to this embodiment will be described with reference to Fig. 3, taking the case where the financial statement is a balance sheet as an example. Fig. 3 is a flowchart showing the flow of processing according to this embodiment. The processing shown in the flowchart in Fig. 3 is executed when the terminal device 30 transmits an image of the financial statement (balance sheet) to the information processing device 10.

In step S101, the acquisition unit 14 acquires one or more images of the financial statement sent from the terminal device 30 via the communication unit 22.

In step S102, the preprocessing unit 15 determines whether the financial statement to be read spans multiple images, i.e., whether the image of a single financial statement to be read is composed of multiple images. If the preprocessing unit 15 determines that the financial statement to be read spans multiple pages, the processing of step S103 is executed next, and if the preprocessing unit 15 determines that the financial statement does not span multiple pages, the processing of step S104 is executed next.

The image of the financial statement to be read may be determined based on the user's selection operation. That is, in the above configuration, if the user selects multiple images, the process of step S103 is executed next, and if the user selects a single image, the process of step S104 is executed next.

In step S103, the preprocessing unit 15 performs a process of synthesizing a single image of a financial statement from multiple images of the financial statement. That is, when the preprocessing unit 15 determines that a single image of a financial statement is composed of multiple images, it synthesizes the single image of the financial statement from the multiple images.

In step S104, the control unit 12 determines the type of financial statement to be read. The type of financial statement may be determined based on a user selection operation. In this example, the control unit 12 determines that the type of financial statement is a balance sheet.

In step S105, the preprocessing unit 15 performs a process of converting the image of the financial statement into a black and white binary image. This process is performed because color information is not particularly required when reading text data.

In step S105, the pre-processing unit 15 performs other processes such as image file format conversion (e.g., conversion from PDF (registered trademark) format to JPG format).
- Processing to clarify text data (adjusting contrast, etc.)
etc. may be carried out.

In step S106, the preprocessing unit 15 determines whether the image of the financial statement contains the character string "composition ratio." "Composition ratio" is an item that is sometimes written in the right column of the amount on the financial statement, and indicates the percentage that the numerical value of the corresponding account item makes up to the total value. If the preprocessing unit 15 determines that the image contains the character string "composition ratio," the processing of step S107 is then executed, and if it determines that the character string is not included, the processing of step S108 is then executed. Note that the specified character string that is the subject of the determination in step S106 is not limited to "composition ratio," and may be any other character string that may be included in the financial statement.

In step S107, the preprocessing unit 15 deletes the column portion including the character string "composition ratio" in the image of the financial statement. Alternatively, in another aspect, the preprocessing unit 15 may be configured to turn on a flag indicating that the image includes the character string "composition ratio" and delete the column portion in step S111 or the like described below.

In step S108, the preprocessing unit 15 performs a process to eliminate the tilt of the image and delete unnecessary parts of the financial statement based on the coordinates of the text data in the image of the financial statement. As a specific example, the preprocessing unit 15 executes the following process.

(1) In the image of the financial statement, the coordinates of the text data that indicates numerical values are calculated. The explanatory diagram 51 in FIG. 4 shows an example in which the text data is framed based on the coordinates of the four corners of each text data.

(2) For the text data whose coordinates have been calculated, pairs of text data are extracted that have a predetermined number (2 or more) of combinations in which the X coordinates in the left-right direction based on the financial statement are the same. As a result, for example, the date in the upper left corner of the financial statement in Figure 4 is excluded from the subsequent processing in this step S108.

(3) For each set of extracted text data, straight lines 59a and 59b are defined along the uppermost numerical values 55a and 55b and the lowermost numerical values 57a and 57b, respectively.

(4) Rotate the financial statement so that the straight lines 59a and 59b defined in (3) above are parallel to the vertical lines 61a and 61b of the entire image.

In step S109, the preprocessing unit 15 deletes, as unnecessary data, data outside the area 67 defined by the numeric values 55b and 57b in the right column of the financial statement and the character strings 63 and 65 of the account items, etc., corresponding to the numeric values 55a and 57a in the left column of the financial statement. The preprocessing unit 15 then divides the financial statement into a right column portion and a left column portion.

In step S110, the preprocessing unit 15 performs processing to at least temporarily change the thickness of at least some of the characters in the text data contained in the image of the financial statement. For example, the preprocessing unit 15 expands the white parts in the image to make the characters thinner while removing noise such as non-character dust, and then contracts the white parts to make the characters thicker, thereby suitably correcting areas where characters are slightly missing due to white inverse noise. In addition, by further expanding the white parts of the thickened characters to make them thinner, it is possible to approach ideal characters with high recognition accuracy. This also makes it possible to extract each piece of text data in a more suitable range.

In step S111, the preprocessing unit 15 extracts the necessary text data based on the height of each bolded text data. Specifically, the preprocessing unit 15 calculates the height of the text data with the most identical height for each column of the divided financial statement, and excludes text data with a height that differs from the calculated height by a predetermined amount or more, or text data with a height that differs from the calculated height by a predetermined amount or more or less, as data unnecessary for subsequent processing. This leaves only the text data that should be read by the reading unit 16 as the target for processing.

In step S112, the preprocessing unit 15 performs a process of rearranging at least a portion of the text data of the financial statement to a position different from the original position. Here, the preprocessing unit 15 may rearrange each piece of text data to a white image different from the original image. This makes it possible to remove noise unrelated to the text data from the image of the financial statement. Figure 5 is an example of a diagram showing a portion of a financial statement in which the preprocessing unit 15 has rearranged the text data. In Figure 5, the start of each line of each account item and each numerical value is aligned, and the line spacing has been adjusted, resulting in an arrangement suitable for reading.

In steps S113 to S115, a process is carried out to classify the categories of account items in the financial statement. Below, an example is given of a process for classifying the account items in the asset section in the left column of the financial statement into the categories of current assets and fixed assets. A similar process is carried out for the right column of the financial statement to classify the account items in the liability section and net assets section.

In step S113, the preprocessing unit 15 determines whether or not it has detected the character string "fixed assets" in the left column of the financial statement. In other words, the preprocessing unit 15 determines whether or not the degree of similarity between any of the text data in the left column of the financial statement and the category name "fixed assets" is equal to or higher than a predetermined value. This is because the account item "current assets" is located above the account item "fixed assets" in the financial statement, and the start position of the "fixed assets" category is the boundary between the categories. If the preprocessing unit 15 detects the character string "fixed assets" in the left column of the financial statement, it then executes the processing of step S114, and if it does not detect it, it then executes the processing of step S115.

In step S114, the preprocessing unit 15 sets the text below the character string "Fixed assets" as an account item belonging to the fixed assets category, and the text up to the character string immediately before the character string "Fixed assets" as an account item belonging to the current assets category. The reading unit 16 also assigns the category name of "Fixed assets" previously registered in the memory unit 20 to the read character string "Fixed assets".

In step S115, the preprocessing unit 15 compares each account item in the left column of the financial statement with the current assets account items registered in the memory unit 20 to determine whether the degree of approximation is greater than or equal to a predetermined value, and sets the start position where the degree of approximation is less than the predetermined value for a predetermined number of consecutive occurrences as the start position of the fixed assets category. The reading unit 16 also assigns the category name "fixed assets" to the text data at the start position.

In step S116, the reading unit 16 reads the text data in the image preprocessed by the preprocessing unit 15. In addition, if the reading unit 16 has assigned a category name in the financial statement to certain text data in step S114 or S115, the reading unit 16 classifies the text data following the text data into the category indicated by the category name and reads it.

The processing of steps S115 and S116 indicates that if the reading unit 16 cannot read text data from the financial statement that has a degree of similarity equal to or greater than a predetermined degree with the first category name preregistered in the memory unit 20, the reading unit 16 reads the start position at which the degree of similarity between the read text data and the account item of the second category is less than a predetermined degree for a predetermined number of consecutive times or more as the start position of the first category.

In step S117, the reading unit 16 compares each piece of data to determine whether the degree of approximation between the target text data and any of the numerical values or accounting items registered in advance in the storage unit 20 and belonging to the category of the text data is equal to or greater than a predetermined value. FIG. 6 is a conceptual diagram showing how the reading unit 16 compares the text data of the financial statement with the accounting items registered in the storage unit 20 by category.

If the reading unit 16 determines that the degree of approximation is equal to or greater than a predetermined value, the process proceeds to step S118. If the reading unit 16 determines that the degree of approximation is less than a predetermined value, the process proceeds to step S119.

In step S118, the reading unit 16 assigns the character string, such as the account item, that has the highest degree of similarity to the target text data.

In step S119, the reading unit 16 assigns the character string read by the mechanism performing OCR processing to the target text data.

In step S120, the presentation unit 17 presents each piece of text data to which a character string has been assigned in step S118 and for which the degree of approximation described above is less than a predetermined value so that the user can modify it. Specifically, the presentation unit 17 generates information showing a screen for modifying the text data in the financial statement, and transmits the information to the terminal device 30 via the communication unit 22. However, the presentation unit 17 may also present each piece of text data for which the degree of approximation described above is equal to or greater than a predetermined value so that the user can modify it.

Figure 7 is a diagram showing an example of the above screen displayed on the terminal device 30 based on information transmitted from the information processing device 10. In the example of Figure 7, the account items of the financial statement read by the reading unit 16 and numerical values indicating the amounts corresponding to the account items are displayed on the screen. The delete button 71 is a button for deleting each item shown in each line. The input item 73 field on the left side of the screen is an object for adding items corresponding to each line of the financial statement.

In addition, on the screen of FIG. 7, each account item and value can be modified. For account items whose degree of approximation is less than a predetermined value, an icon or the like indicating this may be displayed, for example, near the account item. In addition, the current total value of the assets section and the total values of the liabilities section and net assets section are displayed on the screen, but icons or the like may be used to indicate that the above total values are different from each other or that they are the same.

When the user modifies each account item and value appropriately on the screen in FIG. 7 and selects the overwrite button 77, the control unit 12 updates the contents of the financial statement that was read to the contents corresponding to the screen in FIG. 7 in the above case.

When the user selects the register button 79, the calculation unit 18 calculates the judgment result regarding credit management by referring to the items corresponding to each line of the financial statement, i.e., the account items assigned to each text data read by the reading unit 16 and the numerical values corresponding to the account items. The presentation unit 17 then presents the judgment result calculated by the calculation unit 18 to the user via a screen similar to that shown in FIG. 7, etc.

The above describes an information processing method that includes an acquisition step of acquiring an image of a financial statement, a preprocessing step of performing a predetermined preprocessing on the image acquired in the acquisition step according to the type of the financial statement, a reading step of reading text data contained in the image preprocessed in the preprocessing step and assigning a numerical value or a preregistered account item or a category name in the financial statement to each piece of text data based on the degree of approximation, and a presentation step of presenting each piece of text data whose degree of approximation is less than a predetermined value so that the user can modify it. The above method can improve the convenience for users when inputting financial statement data and improve input accuracy.

[Embodiment 2]
A second embodiment of the present invention will be described below. For convenience of explanation, the same reference numerals are given to components having the same functions as those described in the above embodiment, and redundant explanations will not be repeated. In this embodiment, an example in which an information processing device reads a profit and loss statement will be described.

[1. Example of information processing system configuration]
This embodiment also uses the configuration shown in FIG.

2. Examples of processing by the information processing system
The flow of processing in the information processing system 1 according to this embodiment will be described with reference to Fig. 8, taking the case where the financial statement is a profit and loss statement as an example. Fig. 8 is a flowchart showing the flow of processing according to this embodiment. The processing shown in the flowchart in Fig. 8 is executed when the terminal device 30 transmits an image of the financial statement (profit and loss statement) to the information processing device 10.

In steps S101 to S105, the same processing as in embodiment 1 is performed. However, in step S104, the control unit 12 determines that the type of financial statement to be read is an income statement. Following the processing in step S105, the processing in step S206 is performed.

In step S206, the preprocessing unit 15 determines whether the image contains straight lines that will form the borders of the financial statement. If the preprocessing unit 15 determines that the image contains the straight lines, the process proceeds to step S207. If the preprocessing unit 15 determines that the image does not contain the straight lines, the process proceeds to step S209.

In step S207, the preprocessing unit 15 rotates the financial statement so that the vertical straight line that forms the frame line of the financial statement and has the most common angles is parallel to the vertical lines of the entire image.

In step S208, the preprocessing unit 15 deletes data outside the frame that encloses the widest area of the image as unnecessary data.

In step S209, the pre-processing unit 15 performs processing to eliminate the tilt of the image based on the coordinates of text data, such as numerical values, within the image, similar to the processing in step S108.

In step S210, the preprocessing unit 15 deletes data outside the rectangular area surrounding each account item and the numerical values indicating the amounts located at the top left and bottom right as unnecessary data.

In step S211, the preprocessing unit 15 performs a process of relocating at least a portion of the text data of the financial statement to a position different from the original position. As a specific example, the preprocessing unit 15 performs the following process as shown in FIG. 9.

(1) Calculate the coordinates of each piece of text data for each column of the financial statement.

(2) For each column of the financial statements, the process aligns the beginning of the lines of the text data and adjusts the line spacing.

In step S212, the preprocessing unit 15 associates account items and numerical values that have the same vertical y coordinate for the text data in the image.

In step S213, the preprocessing unit 15 determines whether the image contains any of the character strings "composition ratio," "linked," or "previous business." If the preprocessing unit 15 determines that the image contains any of the above character strings, the processing of step S214 is subsequently executed, and if the preprocessing unit 15 determines that the image does not contain any of the above character strings, the processing of step S215 is subsequently executed. However, the character strings to be determined in step S203 are not limited to those exemplified above.

In step S214, the preprocessing unit 15 deletes the string portion that includes the above character string in the image, or a specific string portion that corresponds to the above character string.

In step S215, the reading unit 16 reads the text data in the image that has been preprocessed by the preprocessing unit 15.

In step S216, the reading unit 16 compares the target text data with a numerical value or an account item in the profit and loss statement that has been registered in advance in the storage unit 20 to determine whether the degree of similarity is greater than or equal to a predetermined value.

If the reading unit 16 determines that the degree of approximation is equal to or greater than a predetermined value, the process proceeds to step S118. If the reading unit 16 determines that the degree of approximation is less than a predetermined value, the process proceeds to step S119.

In steps S118 to S120, the same processing as in embodiment 1 is performed. Above, the processing flow based on the flowchart in FIG. 8 has been described.

Even if the type of financial statement is sales and administrative expenses, the preprocessing unit 15 performs preprocessing according to the above type, and then the reading process and other processes are performed. If the type of financial statement is sales and administrative expenses, for example, in the process shown in the flowchart of FIG. 8, the process of step S215 may be performed following the process of step S212.

The preprocessing performed by the preprocessing unit 15 on the image of the financial statement according to the type of financial statement is not limited to the examples described above in each embodiment. For example, the preprocessing unit 15 may be configured to perform processing such as deleting items corresponding to some account items included in a certain type of financial statement.

[Software implementation example]
The control blocks of the information processing device 10 (particularly the acquisition unit 14, the preprocessing unit 15, the reading unit 16, the presentation unit 17 and the calculation unit 18) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

In the latter case, the information processing device 10 includes a computer that executes the instructions of a program, which is software that realizes each function. This computer includes, for example, one or more processors, and a computer-readable recording medium that stores the program. The object of the present invention is achieved by the processor reading the program from the recording medium and executing it in the computer. The processor can be, for example, a CPU (Central Processing Unit). The recording medium can be a "non-transient tangible medium," such as a ROM (Read Only Memory), tape, disk, card, semiconductor memory, programmable logic circuit, etc. The computer can also include a RAM (Random Access Memory) that expands the program. The program can be supplied to the computer via any transmission medium (such as a communication network or broadcast waves) that can transmit the program. One aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. The technical scope of the present invention also includes embodiments obtained by appropriately combining the technical means disclosed in different embodiments.

Reference Signs List 1 Information processing system 10 Information processing device 12 Control unit 14 Acquisition unit 15 Preprocessing unit 16 Reading unit 17 Presentation unit 18 Calculation unit 20 Storage unit 22, 34 Communication unit 30 Terminal device 32 Display unit

Claims (8)

An acquisition unit for acquiring images of financial statements;
a pre-processing unit that performs a predetermined pre-processing on the image acquired by the acquisition unit;
a reading unit that reads text data included in the image preprocessed by the preprocessing unit, and assigns, to each piece of text data, either a numerical value or a preregistered account item or a category name in a financial statement based on a degree of similarity;
a presentation unit that presents each piece of text data whose degree of approximation is less than a predetermined value so that the user can correct the piece of text data;
Account items for financial statements are pre-registered for each category in the financial statements.
The reading unit is
When a category name in a financial statement is assigned to text data included in an image of the financial statement, the following text data is classified into the category indicated by the category name and read.
This information processing device is characterized in that, when text data having a degree of similarity to a pre-registered first category name equal to or greater than a predetermined value cannot be read from a financial statement, the starting position at which the degree of similarity between the read text data and an account item in a second category is less than a predetermined value for a predetermined number of consecutive data is read as the starting position of the first category. The acquisition unit acquires an image of a financial statement uploaded onto a web server,
2 . The information processing apparatus according to claim 1 , wherein the presenting unit transmits information indicating each piece of text data for which the degree of approximation is less than a predetermined value to a terminal device that has uploaded the image onto the web server. 3. The information processing device according to claim 1, further comprising a calculation unit that calculates a judgment result regarding credit management by referring to an account item assigned to each text data read by the reading unit and a numerical value corresponding to the account item. The pre-treatment unit includes:
4 . The information processing apparatus according to claim 1 , wherein at least a part of text data included in the image acquired by the acquisition unit is rearranged to a position different from an original position. The pre-treatment unit includes:
5 . The information processing apparatus according to claim 1 , further comprising a process for at least temporarily changing a thickness of characters of at least a part of text data included in the image acquired by the acquisition unit. The pre-treatment unit includes:
6. The information processing apparatus according to claim 1, wherein, when it is determined that an image of a single financial statement is composed of a plurality of images, an image of the single financial statement is synthesized from the plurality of images . An acquisition step of acquiring an image of a financial statement;
a pre-processing step of performing a predetermined pre-processing on the image acquired in the acquisition step;
a reading step of reading text data included in the image preprocessed in the preprocessing step, and allocating a numerical value or a preregistered account item or a category name in a financial statement to each piece of text data based on a degree of similarity;
a presenting step of presenting each piece of text data whose similarity is less than a predetermined value so that the user can correct it;
Account items for financial statements are pre-registered for each category in the financial statements.
In the reading step,
When a category name in a financial statement is assigned to text data included in an image of the financial statement, the following text data is classified into the category indicated by the category name and read.
An information processing method characterized in that, when text data having a degree of similarity to a pre-registered first category name equal to or greater than a predetermined value cannot be read from a financial statement, the start position at which the degree of similarity between the read text data and an account item of a second category is less than a predetermined value for a predetermined number or more consecutive times is read as the start position of the first category. 7. A program for causing a computer to function as the information processing device according to claim 1, the program causing the computer to function as each of the units.

Images