JP7301671B2 - Image processing device, information processing method and program - Google Patents

Description

The present invention relates to an image processing device, an information processing method, and a program.

There is a business flow in which paper documents such as forms are scanned by an image reading device and digitized. At that time, generally, the digitized document images are stored in a predetermined folder according to the type or contents of the document. Here, manually designating a sorting folder becomes a heavy burden when the number of paper documents to be digitized increases.
As a method for solving this problem, there is a method disclosed in Patent Document 1. In this method, when saving a selected document image, a save destination folder is highlighted (recommended). This recommendation is made using the degree of folder relevance. The folder relevance is a value obtained by calculating similarities between the selected document image and all document images in the folder and averaging the similarities.

JP 2015-76064 A

In this method, recommendations are made based on the document images contained in the folder. Therefore, the recommended folder should have many of the files contained in the folder similar to the selected document image. Therefore, when a folder in which a plurality of types of document images are saved is the save destination folder, appropriate recommendation cannot be made. Such a case is expected to occur, for example, when document images of multiple types of forms are saved in a folder of a worker who performs business processing based on document images. Also, if the storage destination of the document image is determined by information not included in the document image, an inappropriate recommendation may be made. For example, when document images of the same type are stored in different folders for each person in charge of scanning the document images, folders of a plurality of persons in charge are recommended. In such a case, the possibility of selecting the wrong folder due to the user's selection error increases.

According to the present invention, when there is a registered image determined to be of the same type as a document image, an acquisition unit acquires storage destination information regarding a storage destination folder of the registered image; a first determination means for determining whether or not the type of the document image and the storage destination folder are linked; and when the first determination means determines that the type of the document image and the storage destination folder are linked , generating means for generating recommendation information of the storage destination folder; and display means for displaying a folder selection screen containing the recommendation information.

According to the present invention, it is possible to make an appropriate recommendation and reduce the possibility that an incorrect folder is selected due to a user's selection error.

It is a figure which shows an example of the hardware constitutions of an image processing apparatus. 4 is a flowchart showing an example of information processing according to the first embodiment; FIG. 4 is a diagram showing an example of dividing a document image into object blocks; It is a figure which shows an example of the outline|summary of matching. It is a figure which shows an example of a data structure. It is a flowchart which shows the detail of S207. FIG. 10 is a diagram showing a folder selection screen; FIG. FIG. 10 is a diagram showing a folder selection screen; FIG. 10 is a flow chart showing an example of information processing according to the second embodiment; 9 is a flowchart showing details of S901. It is a figure which shows an example of a screen.

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings.

<Embodiment 1>
FIG. 1 is a diagram showing an example of the hardware configuration of an image processing apparatus 100. As shown in FIG.
The image processing apparatus 100 includes a user interface 104, a CPU 105, a RAM 106, a storage section 107, an image reading section 108, an image output section 109, and a display 110, which are connected via the device control section 102 so as to be able to communicate with each other. Further, the image processing apparatus 100 has an image processing unit 103 configured by an ASIC for processing image data. The image processing apparatus 100 is also connected to an external information processing apparatus (not shown) via a network 101 .

The user interface 104 is composed of, for example, a keyboard, mouse, and other input/output devices, and can input various setting values or specified values.
CPU 105 reads the main program from storage unit 107 according to the initial program in storage unit 107 and stores it in RAM 106 . A RAM 106 is used as a main memory for storing programs and for working. Then, the CPU 105 controls the device control section 102 according to the main program. The device control unit 102 is composed of a controller base. Devices such as a CPU 105 , a RAM 106 , a storage unit 107 , an image processing unit 103 and a network 101 are mounted on the device control unit 102 .
The image reading unit 108 is a scanner device, and can acquire a document image in an image data format by reading a paper document or the like. After acquiring the document image from the image reading unit 108 , the device control unit 102 stores it in the storage unit 107 . Then, the device control unit 102 reads the document image stored in the storage unit 107 to the RAM 106 when executing the scan assist processing. Here, the scan assist process is a process of recommending a storage destination folder of a destination.

Then, the device control unit 102 analyzes the document image on the RAM 106 and generates information for scan assist. The analysis process consists of four processes. The first is block selection processing (BS processing) for extracting character string blocks in the document image. The second is optical character recognition processing (OCR processing) for extracting character string information from character string images. The third is a process of determining the type of the document image by obtaining the degree of similarity between the registered document image and the newly scanned document image. The fourth is a process of generating information for scan assist based on the determination process. The information for Scan Assist consists of information related to the save destination folder.

Next, the device control unit 102 generates a UI screen for scan assistance using the generated information for scan assistance, and executes processing for displaying it on the display 110 . Then, the device control unit 102 performs transmission processing based on an instruction from the user via the user interface 104 . Then, the device control unit 102 stores information about the scan assist processing result including feedback from the user in the storage unit 107 .
Further, the device control section 102 provides the document image stored in the storage section 107 to the image output section 109 . The image output unit 109 executes processing for outputting document images in various formats. The image output unit 109 is a printer device. For example, the image output unit 109 can execute processing for outputting image data relating to a document image to a storage medium. Alternatively, the image output unit 109 may have a print function and execute processing for outputting a document image to an output medium such as a paper medium.

The image processing apparatus 100 described above is merely an example, and the image processing apparatus 100 may be configured without the image output unit 109 out of the image reading unit 108 and the image output unit 109 . Also, some functions of the image processing apparatus 100 may be executed by an external processing apparatus by mutual communication via a network. The external processing device may be implemented by a computer device such as a server, or may be implemented by a cloud server on the Internet. The image processing apparatus 100 may have other configurations as required.

Next, information processing according to this embodiment will be described with reference to FIG. FIG. 2 is a flow chart showing the overall scan assist flow for a newly scanned document image in the image processing apparatus 100. As shown in FIG. The flowchart in FIG. 2 is started by receiving a document image scan instruction from the user via the user interface 104 . At that time, the device control unit 102 retains information about the user name that has started up in the RAM 106 . The user name can be acquired by using an authentication device (not shown) or the like. The processing according to the flowchart shown in FIG. 2 is executed by the program code stored in the storage unit 107 being developed in the RAM 106 and the apparatus control unit 102 being controlled by the CPU 105 .

In step S201 of FIG. 2, the device control unit 102 scans a document with the image reading unit 108 and converts it into a document image. Then, the device control unit 102 performs correction processing for the document image, such as color conversion processing and tone correction, in the image processing unit 103 and stores the result in the storage unit 107 . This document image is hereinafter referred to as a query document image.

Next, in step S202, the device control unit 102 reads the query document image stored in the storage unit 107 to the RAM 106, and performs rotation correction processing. The device control unit 102 performs rotation correction processing by calculating the rotation angle and rotating the image using the calculated rotation angle. The device control unit 102 calculates the rotation angle using the fact that the character strings and lines in the document image are digitally arranged in the horizontal direction. For example, the device control unit 102 can calculate the rotation angle by taking projection histograms in various directions and selecting angles corresponding to histograms in which the peak and bottom of the histogram oscillate greatly in a short period. . This is because, if projected in the correct direction, horizontal lines such as character strings in the same line and ruled lines in the same direction are voted in the same bin on the histogram, and nothing is voted in the space between lines. This is because a large amplitude occurs in the period between characters. The angle calculated by the method up to this point does not consider the orientation of the character, and there is an uncertainty of 180 degrees. The device control unit 102 can determine the orientation of characters by using character likelihood information obtained when a simple character recognition process is performed. As a result, it is possible to calculate the angle information in consideration of the orientation of the characters.

Next, in step S203, the device control unit 102 executes block selection processing (BS processing) on the query document image. The block selection process is a process of dividing an area in an image into object blocks and determining attributes of each block. A specific example will be described with reference to FIG. FIG. 3A is an example of the document image read in step S201. FIG. 3B shows the document image divided into object blocks. Attributes such as TEXT/PICTURE/PHOTO/LINE/TABLE are determined for each block, and divided into areas having different attributes.

Here, a method of block selection processing will be described below. First, the apparatus control unit 102 binarizes the document image into black and white, performs contour tracing, and extracts a group of pixels surrounded by black pixel contours. For a mass of black pixels whose area is larger than a predetermined area, the device control unit 102 also performs contour line tracing on white pixels inside, extracts a mass of white pixels, A block of black pixels is recursively extracted from inside the block of pixels.

The apparatus control unit 102 classifies the clusters of black pixels obtained in this way by size and shape, and classifies them into regions having different attributes. For example, the device control unit 102 determines pixel clusters having an aspect ratio close to 1 and a size within a certain range as pixel clusters corresponding to characters. Further, the device control unit 102 sets a portion where adjacent characters can be grouped in good order as a character area (TEXT). The device control unit 102 sets the flat pixel block as a line area (LINE). The device control unit 102 defines a table area (TABLE) as a range occupied by black pixel blocks having a certain size or larger and including rectangular white pixel blocks in a well-aligned manner. The device control unit 102 sets the area where irregular pixel blocks are scattered as a photo area (PHOTO). Then, the device control unit 102 sets the other arbitrary-shaped pixel block as a picture area (PICTURE). The attribute information for each block obtained by the block selection process is used as information for similarity calculation, etc., which will be described later. Here, a block determined as a character area is called a BS text block. Information in BS text blocks is also available for OCR (Optical Character Recognition) processing. The device control unit 102 may perform OCR processing as necessary in step S203.

Next, in step S204, the device control unit 102 performs matching processing between the query document image and the group of registered document images to determine the type of the document image. Here, the registered document image group is a collection of previously processed document images registered in step S212, which will be described later. The matching process is a process of calculating similarities between the query document image and all registered document images and obtaining a registered image having the maximum similarity. When the maximum degree of similarity is equal to or greater than a predetermined threshold, the device control unit 102 determines that the query document image is of the same type as the registered document that has obtained the maximum degree of similarity. If the maximum degree of similarity does not exceed a predetermined threshold, the device control unit 102 determines that there is no registered image of the same type. The device control unit 102 reads out the registered document image group from the storage unit 107 to the RAM 106 and uses it. Also, the registered document image group may not be the document images themselves, but may be feature amounts that can be used for matching processing such as block selection processing results of each document image.

Here, an overview of matching will be described with reference to FIG. FIG. 4A is a query document image, and FIGS. 4B and 4C are registered document images. 4A and 4B are similar in text placement, and FIGS. 4A and 4C are dissimilar in text placement. Therefore, if the similarity between FIGS. 4A and 4B and the similarity between FIGS. 4A and 4C are calculated using an appropriate method, the similarity of the former is is higher. Therefore, in this case, FIG. 4B is selected as a matching result.

Here, the concept of similarity calculation using BS text blocks will also be explained. FIG. 4(D) illustrates the BS text block of FIG. 4(A). The dotted lines are BS text blocks, and ID000 to ID004 are the IDs of the respective text blocks. It can be seen that the character string in FIG. 4A is a BS text block. Similarly, FIG. 4(E) illustrates the BS text block of FIG. 4(B), and FIG. 4(F) illustrates the BS text block of FIG. 4(C). Similarity calculation using BS text blocks is a method of calculating similarity by focusing on how similar the shapes and arrangements of BS text blocks are. 4(D) and 4(E) have many similar BS text blocks. Specifically, FIG. 4(D) ID000 and FIG. 4(E) ID004, FIG. 4(D) ID001 and FIG. 4(E) ID003, FIG. 4(D) ID002 and FIG. 4(E) ID000, FIG. D) ID003 and FIG. 4(E) ID001 are similar. Furthermore, FIGS. 4(D) and 4(E) are aligned so that FIG. 4(D) ID004 and FIG. 4(E) ID005 are at the same position. 4(D) ID004 and FIG. 4(E) ID005, FIG. 4(D) ID003 and FIG. 4(E) ID002, and FIG. 4(D) ID002 and FIG. 4(E) ID001 are similar. In FIGS. 4(D) and 4(F), BS text blocks slightly overlapping exist, but BS text blocks with high similarity do not exist. Therefore, when calculating the similarity between FIGS. 4(D) and 4(E) and the similarity between FIGS. 4(D) and 4(F), the former has a higher similarity.

Here, the matching using the BS text block, which is the form layout information, has been described. However, the device control unit 102 may also utilize other information to perform matching and determine the degree of similarity between document images. For example, the device control unit 102 utilizes user information (personal ID, organization ID, etc.), OCR information (OCR result of the entire document, presence/absence of a specific keyword, personal name, company name, title information, etc.) to determine the degree of similarity. may be determined. In addition, the device control unit 102 uses a plurality of similarity determination logics to find registered document images that are determined to be of the same type from a plurality of viewpoints, and determines the type and storage destination of each document image in a later step. It may be determined whether the folders are linked.

Subsequently, in step S205, the device control unit 102 determines whether or not there is a registered document determined to be of the same type in step S204. If there is a registered document determined to be of the same type, the device control unit 102 proceeds to step S206, and if there is no registered document determined to be of the same type, proceeds to step S211.
When the process proceeds from step S205 to step S206, the device control unit 102 performs a process of acquiring storage destination folder information corresponding to registered document images determined to be of the same type obtained in step S204. This storage destination folder information is information relating to the storage destination of the past query document image group determined to be of the same type as this registered document image. The information includes the folder path (including URL etc.) of the storage destination folder, the date of storage, the information of the person who saved, the folder path of the recommended storage destination, information as to whether the recommendation was valid, and the like.

Here, an example of the data structure held in the RAM 106 or the storage unit 107 by the device control unit 102 will be described with reference to FIG. FIG. 5A shows a data structure for holding BS text block information (process result of step S203) of the query document image. The first column is information about the variables that this data structure has, and the second column is information about the type of each variable. Here, "short" in the type column indicates an integer type. "[ ]" indicates a vector type. "floAt" indicates floating point. "BooleanAn" indicates a Boolean type. "String" indicates a character string. "Date" indicates a date type. This is common in the drawing of FIG. In FIG. 5A, the query document image has a vector data structure of "BS text block" in addition to the information of image width, image height, and number of BS text blocks. The data structure of the "BS text block" is shown in FIG. 5(B). A "BS text block" has a block ID, a block width, a block height, and the X and Y coordinates of the upper left corner of the block.

FIG. 5C shows the data structure of the registered document image. This is generated in step S212, which will be described later. Specifically, it consists of a registered document image ID, an image width, an image height, the number of BS text blocks, a "BS text block" vector, and a "storage destination folder information ID." Here, "BS text block" means the data structure of FIG. 5(B). The device control unit 102 performs matching processing between the query document image and the registered document image using the information of the BS text block lists of FIGS. 5A and 5C. The data structure for storing the matching result of this matching process is shown in FIG. 5(D). The matching result has the ID of the matched registered document and the degree of similarity calculated by the matching process.

The “storage destination folder information ID” in FIG. 5C is an ID indicating an instance of the data structure of the storage destination folder information corresponding to the registered document image. A specific data structure of the destination folder information is shown in FIG. 5(E). The storage destination folder information has a storage destination folder information ID, a corresponding registered document image ID, and a vector of "individual query document image storage destination information". The corresponding registered document image ID is the ID of the registered document image corresponding to the instance of the destination folder information. The data structure of the “individual query document image storage location information” is shown in FIG. 5(F). The save destination folder path is a character string that uniquely represents the save destination folder. This character string may be a character string suitable for the storage destination system, such as a relative path from the root folder of the storage destination system, or a URL. The save date is the date when the query document image was sent and saved. The saved user name is the name of the user who sent the query document image. The saved user name can be obtained from the user name of the user who started the flowchart of FIG. Each element of the recommendation success flag list becomes "true" when the storage destination folder recommended in step S210, which will be described later, is used by the user. If not used, it becomes "false". It is a vector because it may recommend multiple folder paths. The actually recommended folder path becomes an element of the recommendation destination folder path list. It is a vector because it may recommend multiple folder paths. Information indicating whether the recommendation of the x-th element of the recommendation destination folder path list was successful becomes the x-th element of the recommendation success flag list.

FIG. 5G shows a data structure used in folder recommendation information, which will be described later. The element of the recommendation destination folder path list is the path of the storage destination folder to be recommended. The 0th element of the recommendation destination folder path list becomes the path of the storage destination folder with the highest recommendation priority, and thereafter, the paths of the storage destination folders are held in the recommendation destination folder path list in order of priority.
Assuming the data structure as described above, the steps to be described later will be described.

Subsequently, in step S207, the device control unit 102 determines whether or not the registered document image determined to be of the same type is associated with the storage destination folder based on the acquired storage destination folder information. Here, the detailed processing of this determination will be described using the flowchart of FIG. FIG. 6 is a flowchart showing the details of the processing in step S207. First, in step S501, the device control unit 102 calculates a statistic from the destination folder information. The statistic here is a statistic for determining whether the registered document image and the save folder are linked. Here, "associated" means that the user selects a storage folder according to only the type of document image. If it is "associated", it means that the storage destination folder is recommended according to the type of the document image. Examples of statistics are shown below.

・Most frequent folder path save count A
・ Total save count B
・Most frequent folder saving ratio C = A/B
The save count A of the most frequent folder path is the total save count of F folder paths with the highest save counts in the save destination folder information. If it is assumed that the document image type and the save destination folder are statically associated, F may be 1. In order to cope with the case where the save destination folder is periodically changed, it is necessary to set F larger than 1, which will be described in detail later. The total storage count B is the number of pieces of individual storage information corresponding to the query document image group held by the storage destination folder information. These statistics can be calculated if the data structures shown in FIGS. 5(E) and 5(F) are held. The storage count A of the most frequent folder path can be calculated by summing up the storage destination folder paths shown in FIG. 5(F). The total storage count B is the number of vectors of the individual query document storage destination information in FIG. 5(E). Here, when calculating the number of saves A and the total number of saves B of the most frequent folder path, weighting is performed so that the most recent saves are emphasized, or only the save destination folder information within a certain period of time is calculated. may be used. By doing so, it is possible to improve the responsiveness to the case where the save destination folder is changed. For example, in a case where the storage folder is changed periodically, the most frequent folder storage ratio C decreases as time elapses. In such a case, if F is set to 2 and only information saved in a period equal to or shorter than the change interval is collected, only the information of the current or previous folder is always used, and the main folder storage ratio C is calculated. Therefore, the most frequent folder saving ratio C can be kept high. However, if F is set to be greater than 1, it is necessary to strictly set the coefficients of the determination process, which will be described later, in order to avoid wrong linking.

Subsequently, in step S502, the apparatus control unit 102 determines the link between the type of form and the save destination folder. The statistic calculated in step S501 is used for the determination.
For example, the device control unit 102 makes the following determinations.
・Condition 1: Most frequent folder storage ratio C>N
・Condition 2: Total save count B>M
・Judgment condition: Satisfying both conditions 1 and 2 If N is set to 0.8 and M is set to 2, etc., folder paths that are saved more than 80% of layouts that are saved to folders more than 3 times Only when there is a group, it will be determined that they are linked. Note that if N is set to 0.8 only under condition 1, it will always be determined to be linked at the second time, resulting in an inappropriate determination. It should be noted that the above-described F and N may be switched according to the total number of times B of storage so that the determination can be made stable. For example, when the total number of times of storage is small, the reliability of the statistics is low. Therefore, if F is set small and N is set large, it is possible to determine the link only when the link is definitely linked.

Conversely, there are cases where it is desirable to implement recommendations early. For example, for a user who always has correspondence between document image types and storage folders, it is stressful that the same type of document image cannot be recommended unless the same type of document image is used three or more times. For example, when the weighted average of the most frequent folder storage ratio C (weighted by the number of times of storage) in all registered document image groups exceeds a certain percentage, the type of document image always corresponds to the storage folder. is assumed. In such a case, it is assumed that the new document image is also supported. Therefore, the device control unit 102 may, for example, set N: 0.8, M: 0, etc., so that recommendations are made from the second time.
Further, when the user explicitly instructs, the device control unit 102 may always perform the linking determination. The user's explicit instruction may be the user's instruction when the flowchart of FIG. 2 is first activated, or the user's instruction when performing presetting related to the flowchart of FIG.

The above is the details of the processing in step S207. Note that the processing in step S207 can be calculated in advance. In order to improve the processing performance, the device control unit 102 may hold the results of pre-calculations in the storage unit 107 and refer to the results.

Subsequently, in step S208, the device control unit 102 branches the processing based on the determination result in step S207. If the device control unit 102 determines in step S207 that the document image type and the save destination folder are linked, the process proceeds to step S209. If the device control unit 102 does not make a determination, the process proceeds to step S211.
When proceeding to step S209, the device control unit 102 generates folder recommendation information. The folder recommendation information is information about a destination folder to be recommended to the user. An example of a specific data structure is the aforementioned FIG. 5(G). As a folder to be recommended, the following folders are assumed.

・Folder in which document images of the same type were most recently saved ・Folder in which most document images of the same type were saved in the most recent R times ・Folder in which most document images of the same type were saved・The folder in which the user executing the flowchart saved the most document images of the same type in the last R times ・The folder in which the user executing the flowchart saved the most document images of the same type folder with many saves

Here, R is a preset constant. These folders can be found using the save destination folder information. Specifically, it can be obtained by using the save destination folder path, save date, and save user name in FIG. 5(F). Among these recommendation candidates, which folder should be used and what order of priority should be given are set by the user according to the assumed usage pattern, set in advance by the user, weighted according to the success history of recommendations, etc. , there are various methods. For example, if the rules for determining the storage destination do not depend on the processing user, and the storage destination folder may change periodically, recommendations can be made in the following format.

1st priority: the folder in which the same type of document image was saved most recently 2nd priority: the folder in which the same type of document image was saved the most in the most recent R times In the case of , only the folder with the first priority is recommended. The reason for this recommendation is that there is a possibility that the folder that was saved most recently will be used from the next time onwards, as there is a possibility that it will be changed periodically. Because it is expensive. However, since there is also the possibility of user's erroneous operation, the folder most saved in the most recent R times is also recommended.
Since the link determination in step S207 and the folder recommendation information generation in step S209 are separated, unlike the link determination, recommendations can be made from the perspective of the most recently saved folder in addition to the number of saves. . In other words, it is possible to generate folder recommendation information from a viewpoint different from link determination.

Subsequently, in step S<b>210 , the device control unit 102 generates a folder selection screen with folder recommendation based on the folder recommendation information, and displays it on the display 110 . Then, the user interface 104 receives an operation by the user. The folder selection screen with folder recommendation only needs to be able to easily select the folder path included in the folder recommendation information. If the user selects a destination folder, the device control unit 102 transitions to step S213.
Here, an example of UI will be described with reference to FIG. The screen shown in FIG. 7 is displayed on display 110 . User operations are performed using the user interface 104 . In addition, the device control unit 102 can appropriately acquire the folder information of the preset destination server or storage via the network 101 .

FIG. 7A shows a folder selection screen when there is only one save destination folder to be recommended. That is, this is the folder selection screen when there is only one element in the recommendation destination folder path list of FIG. 5(G). In FIG. 7A, the storage destination folder to be recommended is "layer 3_ABD". In FIG. 7A, a folder is recommended by displaying this folder selectable screen from the beginning. In the selection candidate folder display field 601, "hierarchy 1_A/hierarchy 2_AB/hierarchy 3_ABD" is displayed. "Layer 1_A/Layer 2_AB/" is a folder path including upper folders. "Layer 3_ABD" is a folder name. Then, in the folder content display field 602, the content of “hierarchy 3_ABD” is displayed. In this example, there is no folder in "hierarchy 3_ABD", and only a button 603 for returning to the upper hierarchy folder is displayed. A button 604 is a button for returning to the screen before transitioning to this UI screen. A button 605 is a button for selecting a folder, and when selected, "hierarchy 3_ABD" is selected as the save destination folder. When the user wants to select an arbitrary folder, he uses the folder contents display field 602 to move to an appropriate folder and selects a button 605 . On this screen, first, a button 603 is selected to move to a higher-level folder, and from there, move to an appropriate folder.

FIG. 7B is a folder selection screen when there are a plurality of storage destination folders to be recommended. In other words, this is the folder selection screen when there are a plurality of elements in the recommendation destination folder path list of FIG. 5(G). The difference between FIG. 7B and FIG. 7A is the presence of button 606 . In this example, when the button 606 is selected, the screen transitions to the screen shown in FIG. 7(C). In FIG. 7C, the recommended storage destination folder is "Layer 3_ABC", and the display of the selection candidate folder display column 607 is also switched to "Layer 1_A/Layer 2_AB/Layer 3_ABC". When the button 608 is selected, "Layer 3_ABC" is selected as the save destination folder.
When the button 609 is selected, the screen transitions to a screen for recommending the next element in the recommendation destination folder path list. The screen for recommending the last element of the recommendation destination folder path list transitions to the screen for recommending the first element.

Furthermore, another UI example will be described with reference to FIG. The screen shown in FIG. 8 is displayed on the display 110 and is operated by the user using the user interface 104 . In addition, the device control unit 102 can appropriately acquire the folder information of the preset destination server or storage via the network 101 .

FIG. 8A shows a folder selection screen when there are two storage destination folders to be recommended. That is, this is the folder selection screen when there are two elements in the recommendation destination folder path list of FIG. 5(G). In FIG. 8A, storage destination folders to be recommended are "hierarchy 3_ABD" and "hierarchy 3_ABC". Corresponding to these, the folder selection button 701 displays "layer 1_A/layer 2_AB/layer 3_ABD", and the folder selection button 702 displays "layer 1_A/layer 2_AB/layer 3_ABC". When the folder selection button 701 is selected, "layer 3_ABD" is selected as the save destination folder. When the folder selection button 702 is selected, "layer 3_ABC" is selected as the save destination folder. A button 703 is a button for returning to the screen before transitioning to this UI screen. When a button 704 is selected, the screen transitions to FIG. 8B, which is a screen for the user to select an arbitrary folder. A selection candidate folder display field 705 in FIG. 8B displays the folder name of the current selection candidate, and displays "root" which is the folder name of the highest layer. A folder content display field 706 displays a button for moving to a folder included in the "root" folder. By sequentially selecting this button, the user can move to any folder. By selecting the button 707 after moving to the folder that the user wants to select, the corresponding folder can be selected.

Next, a description will be given of the transition from step S205 or step S208 to step S211. In step S<b>211 , the device control unit 102 generates a folder selection screen without folder recommendations and displays it on the display 110 . Then, the user interface 104 receives an operation by the user. The folder selection screen without folder recommendation should allow the user to easily select a desired folder. When the user selects a destination folder, the device control unit 102 transitions to step S212. An example of the UI may be the same as that of FIG. 8B, so description thereof will be omitted.

In step S212, the device control unit 102 updates information for document matching. If there is a transition from step S205 to step S211, it means that there is no matching document image, so it is necessary to register the query document image as a new registered document image. In that case, the query document image is registered as a new registered document image. Registration is performed by device control section 102 storing registration information in storage section 107 . Information to be registered is information necessary for matching. Information required for matching is the BS processing result of the query document image calculated in step S203 in this embodiment. This is because the information on the BS processing result includes information on the structure of the query document image and is useful information for matching. Note that the device control unit 102 may store the image information of the query document image itself, or another type of known feature amount such as a local feature amount, and use it for document image matching. A specific example of the data structure is the data structure of the registered document image shown in FIG. 5C.

After transitioning from step S210 or step S212 to step S213, the device control unit 102 updates the destination folder information. The device control unit 102 updates the storage destination folder information corresponding to the registered document image determined to be matched in step S205 or the registered document image newly registered in step S212. The content of the update is the addition of information about the save destination folder finally selected by the user in step S210 or step S211. The information includes, for example, a folder path (including a URL, etc.), a save date, information on a user executing the flowchart, and the like. When transitioning from step S210, the device control unit 102 also adds information such as the folder path of the storage destination of the recommendation and whether or not the recommendation was valid. Whether or not the recommendation is valid can be determined by comparing the folder finally selected by the user and the recommended storage destination folder. An example of a specific data structure is the storage destination folder information in FIG. 5(E) and the individual query document storage destination information in FIG. 5(F). It will be newly added. The correspondence between the registered document image and the storage destination folder information is possible by entering the same ID in the storage destination folder information ID of FIG. 5C and the storage destination folder information ID of FIG. 5E. . Furthermore, in order to enable reference in the opposite direction, the same ID may be entered in the registered document image ID of FIG. 5C and the corresponding registered document image ID of FIG. 5E.

Finally, in step S214, the device control unit 102 sets transmission of the query document image. The transmission settings are generated based on the information on the destination folder selected in step S210 or S211 and the information on the preset destination server or storage. Then, the device control unit 102 transmits the query document image via the network 101 according to the transmission setting.

The above is the flow chart of the information processing in the image processing apparatus 100 . Since steps S214 and S213 have no dependency, they may be executed in parallel or their order may be changed. As described above, the series of information processing in this embodiment uses the correspondence between the document image type and the destination folder selected by the user. Recommendations can be made without being influenced. Therefore, even if a folder in which a plurality of types of document images are stored is a storage destination folder, appropriate recommendations can be made. In addition, even if the document image is moved from the primary storage destination folder according to the work flow after sorting, appropriate recommendation can be made.

In addition, in the series of information processing according to the present embodiment, it is determined whether or not the storage destination folder is determined based only on the information included in the document image. It is possible not to make a recommendation when In other words, by not making inappropriate recommendations, it is possible to reduce the possibility of the user selecting an incorrect folder due to a selection error.

In this embodiment, BS text blocks are used for similarity calculation, but the device control unit 102 may use other similarity calculation methods. For example, in order to compare the appearance of the images, the device control unit 102 may calculate the degree of similarity using the pixel values of the low-resolution document images. At this time, the device control unit 102 uses the sum of the absolute values of the differences in the luminance values of the document images or the correlation coefficient as the degree of similarity. Alternatively, the device control unit 102 may calculate the degree of similarity using the character strings of the OCR results in order to see the degree of similarity of the contents of the sentences. In this case, the device control unit 102 may obtain a similarity scale using the Levenshtein distance or the like, which is a scale for calculating the distance between character strings.

<Embodiment 2>
The second embodiment will be described below with respect to portions different from the first embodiment.
In the second embodiment, a method for reducing the user's manual setting work when the type of document image and the folder are not directly linked will be described.
The difference from the first embodiment is the flowcharts of FIGS. 2 and 6, so the flowchart of FIG. 9 is referred to instead of the flowchart of FIG. 2, and the flowchart of FIG. 10 is referred to instead of the flowchart of FIG. do.

First, the flowchart of this embodiment will be described with reference to FIG. FIG. 9 is a flow chart showing the overall scan assist flow for a newly scanned document image in image processing apparatus 100 . The starting method and execution method of the flowchart of FIG. 9 are the same as those of the flowchart of FIG.
2 and 9 are steps S207 and S901, steps S209 and S902, and steps S210 and S903. Therefore, only steps S901, S902, and S903 will be described.

In step S<b>901 , the apparatus control unit 102 determines whether the registered document image determined to be of the same type is associated with the storage destination folder based on the acquired storage destination folder information. Here, detailed processing of this determination will be described using the flowchart of FIG. 10 . FIG. 10 is a flowchart showing the details of the processing in step S901. First, since steps S501 and S502 are the same as steps S501 and S502 in FIG. 6, the description thereof is omitted.

In step S1001, the device control unit 102 initializes to 1 a variable P that is used in subsequent steps. It should be noted that this variable P is retained in the RAM 106 even after the flowchart of FIG. 10 is completed because it will be used in subsequent steps.
Subsequently, in step S1002, the device control unit 102 branches depending on whether it is determined in step S502 or S1004 that the type of form and the save destination folder are linked. The device control unit 102 advances to step S1006 when the connection is determined. The device control unit 102 advances to step S1003 when the connection determination is not made.

When the process proceeds from step S1002 to step S1003, the device control unit 102 calculates the statistical information after setting the folder path included in the save destination folder information to the P layer.
For example, if the folder path included in the folder information is "layer 1_A/layer 2_AB/layer 3_ABD"
"Layer 1_A/Layer 2_AB/Layer 3_ABB"
"Layer 1_A/Layer 2_AB/Layer 3_ABC"
"Layer 1_A/Layer 2_AA/Layer 3_ABA"
Suppose it was In this case, if P was 1,
"Layer 1_A/Layer 2_AB"
"Layer 1_A/Layer 2_AB"
"Layer 1_A/Layer 2_AB"
"Layer 1_A/Layer 2_AA"
is a path on the P hierarchy. The folder paths were all different before being placed on the P layer, but by moving to the P layer, 3/4 of the folder paths become the same folder path. For the user, if this folder path is recommended, the folder can be set with fewer man-hours than manually setting the save destination folder from the top folder. Using this folder path on the P hierarchy, the device control unit 102 calculates the statistic in the same manner as in step S502. Note that a folder path whose layers up to the highest layer are P layers or lower is treated as the highest layer.

Subsequently, in step S1004, the device control unit 102 determines the link between the type of form and the destination folder. Except that the statistic used in this step is the statistic calculated in step S1003, the process is the same as step S502 in FIG.
Then, in step S1005, the device control unit 102 adds 1 to the variable P. This is done in order to try whether or not a link determination is made with a folder path of a higher layer when the link determination is not performed with the folder path on the P layer.
When the process proceeds from step S1002 to step S1006, the device control unit 102 determines whether the folder path with the largest number of times of saving is the folder of the highest layer when the association is determined. As P becomes larger, all folder paths eventually become the highest layer, and this determination is made because it becomes meaningless determination of linking. If the device control unit 102 does not determine that the folder is the highest hierarchical folder, the device control unit 102 terminates the processing of the flowchart. If it is determined to be the folder of the highest hierarchy, the device control unit 102 proceeds to step S1007.
When the process proceeds from step S1006 to step S1007, the device control unit 102 changes the linking determination to determine that there is no linking.

The above is the details of the processing in step S901. It should be noted that the processing in step S901 can be calculated in advance. In order to improve the processing performance, the device control unit 102 may hold the results of pre-calculations in the storage unit 107 and refer to the results.

Next, step S902 will be described. In step S902, the device control unit 102 generates folder recommendation information. Folder recommendation information is the same information as in the first embodiment. When the variable P generated by the information processing in FIG. 10 is 1, the device control unit 102 performs the same processing as in step S209 in FIG. If the variable P is greater than 1, the device control unit 102, in step S1003 of FIG. 10, sets the folder path with the largest number of saves as the save destination folder to be recommended to the user. However, this save destination folder is based on the premise that the user manually sets the final save destination folder based on this folder.

Finally, step S903 will be explained. In step S<b>903 , the device control unit 102 generates a folder selection screen with folder recommendation based on the folder recommendation information, and displays it on the display 110 . Then, the user interface 104 receives an operation by the user. The folder selection screen with folder recommendation only needs to be able to easily select the folder path included in the folder recommendation information. After the user selects a destination folder, the device control unit 102 transitions to step S213.

The UI example here is the same as in the first embodiment when the variable P is 1. A case where the variable P is greater than 1 will be described with reference to FIG. The screen shown in FIG. 11 is displayed on the display 110 and the user's operation is performed using the user interface 104 . Further, the device control unit 102 can appropriately acquire folder information of a preset destination server or storage via the network 101 .

In FIG. 11, the storage destination folder to be recommended is "layer 2_AB". In FIG. 11, a folder is recommended by displaying a screen on which this folder can be selected from the beginning. However, since the variable P is actually greater than 1, there is a high possibility that the folder that the user wants to save is a folder in the lower layer of this folder. In the selection candidate folder display field 1101, "hierarchy 1_A/hierarchy 2_AB" is displayed. "Hierarchy 1_A/hierarchy 2_AB/" is the folder path including the upper folder, and "hierarchy 2_AB" is the folder name. Then, in the folder content display column 1102, the content of “hierarchy 2_AB” is displayed. In this example, there are a plurality of folders in "hierarchy 2_AB", and buttons for moving to each folder are arranged. A button 1103 is a button for returning to the screen before transitioning to this UI screen. When the user moves to an appropriate folder using the folder content display column 1102 and selects a button 1104, that folder is selected as the save destination folder.

As described above, the series of information processing according to the present embodiment has the effect of the first embodiment, and in addition, even when the type of the document image and the storage destination folder are not directly linked, they can be used in common. higher-level folders are recommended. As a result, compared to manually selecting folders from the highest hierarchy folder, the user only needs to select folders in the middle hierarchy, so that it is possible to reduce the trouble of folder selection. Especially when the folder hierarchy is deep, or when the number of folders is large, the reduction effect is large.
Note that a button such as "recommend middle layer" may be provided on the folder selection screen in step S211 of FIG. 2 according to the first embodiment to transition to step S901 of FIG. 9 according to the present embodiment.

<Other embodiments>
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or storage medium. It can also be realized by a process in which one or more processors in the computer of the system or device reads and executes the program. It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

Although one example of the embodiment of the present invention has been described in detail above, the present invention is not limited to such a specific embodiment.

As described above, according to each of the above-described embodiments, since the relationship between the document image storage destination specified by the user and the type of the document image is used, many of the document images already included in the folder can be replaced with the new folder. It does not have to be similar to the document image to be sorted. Therefore, even if a folder in which a plurality of types of document images are stored is a storage destination folder, appropriate recommendations can be made. In addition, since the information on the storage destination folder of the distribution destination is held, even when the document image is moved from the primary storage destination folder according to the work flow after the distribution, appropriate recommendation can be made.
In addition, since it is also determined whether the save destination folder is determined based only on the information contained in the document image, if the save destination of the document image is determined by information other than the document image itself, no recommendation is made. can be done. In other words, by not making inappropriate recommendations, it is possible to reduce the possibility of the user selecting an incorrect folder due to a selection error.

100 image processing device 102 device control unit

Claims (8)

acquisition means for acquiring storage destination information regarding a storage destination folder of the registered image when there is a registered image determined to be of the same type as the document image;
a first determination means for determining whether or not the type of the document image and the storage destination folder are linked based on the storage destination information;
generating means for generating recommendation information for the storage destination folder when the first determination means determines that the type of the document image and the storage destination folder are linked;
display means for displaying a folder selection screen including the recommendation information;
An image processing device comprising: 2. The image processing apparatus according to claim 1, further comprising updating means for updating said storage location information based on information relating to the folder selected on said folder selection screen. 3. The image processing apparatus according to claim 1, wherein when there is no registered image determined to be of the same type as the document image, the display means displays a folder selection screen that does not include recommendation information. . 3. The method according to claim 1, wherein when the type of the document image and the storage destination folder related to the storage destination information are not linked, the display means displays a folder selection screen that does not include recommendation information. Image processing device. further comprising second determination means for determining whether or not there is a registered image determined to be of the same type as the document image;
When the second determining means determines that there is a registered image determined to be of the same type as the document image, the acquiring means acquires storage destination information regarding a storage destination folder of the registered image. 5. The image processing apparatus according to any one of claims 1 to 4, characterized by: The first determination means determines whether or not the type of the document image and the storage destination folder are linked based on the number of times of storage to the storage destination folder obtained from the storage destination information. 6. The image processing apparatus according to any one of claims 1 to 5. An information processing method executed by an image processing device,
an acquisition step of acquiring storage destination information regarding a storage destination folder of the registered image when there is a registered image determined to be of the same type as the document image;
a first determination step of determining whether the type of the document image and the storage destination folder are linked based on the storage destination information;
a generation step of generating recommendation information for the storage destination folder when the first determination step determines that the type of the document image and the storage destination folder are linked;
a display step of displaying a folder selection screen including the recommended information;
An information processing method comprising: A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 6 .

Images