JP6158736B2 - Information processing apparatus, information processing system, and information processing program - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing system, and an information processing program, and more particularly to an information processing apparatus, an information processing system, and an information processing program that can change image data into document data.

2. Description of the Related Art Conventionally, there are image forming apparatuses (information processing apparatuses) such as multifunction peripherals (MFPs) capable of printing documents and images.
Some of these image forming apparatuses have a function of converting image data acquired by a scanner or the like into document data such as PDF (Portable Document Format).

  As an example of such an apparatus, referring to Patent Literature 1, attribute separation means for extracting attribute data including at least a character attribute from image data, and character attributes among attribute data extracted by the attribute separation means, Vectorization that determines which character font is stored in advance, vectorizes it using the outline corresponding to the determined font, and vectorizes by taking the outline as a line drawing for attributes other than character attributes There is disclosed an image processing apparatus comprising: a processing unit; and a transmission unit that transmits vectorized attribute data vectorized by the vectorization processing unit together with image data to another device.

JP 2009-284171 A

However, since the technique of Patent Document 1 only vectorizes a character, it cannot acquire font information such as the font type and character size of the character.
That is, since the font information when the image data is created by word processor software or the like is unknown, the image data cannot be converted into the same document data as the original document.

  The present invention has been made in view of such a situation, and an object thereof is to provide an image forming apparatus that solves the above-described problems.

The information processing apparatus of the present invention, the image data obtained in the information processing apparatus to change the document data output, and the image data, storage means for storing a font matching data of the plurality of fonts and font size font , a character recognizing section for recognizing characters from the image data stored in the storage means, the area of the image data is recognized as the character by the character recognition means, it is recognized the character, in the storage means wherein a font matching means for matching or corresponding to any of the stored plurality of fonts and a plurality of character sizes of the font in the verification data, the font and the character size is matched with the font verification unit as character attribute data set for the document data, and, the sentence so as to correspond to drawing position location of the characters on the image data And a character attribute data setting means for setting as attribute data to the document data, the character attribute data setting means (1) collated the font and said character size, the document of the font in the matching data Compare the image of the character with the image of the character, calculate the best matching coordinate, perform an origin calculation process to calculate the origin of the character based on this coordinate, and (2) within the collated font verification data Designating the font and the character size, and at least a bounding box that is a drawing area of the character of the text data, and the origin of the character and the origin of the next character calculated in the origin calculation process Drawing data including a next origin, a side bearing of the character, and a character width of the character is used as the character attribute data for the document data. A character drawing position correction setting process to be set is performed. (3) For adjacent characters in the document data, it is determined whether or not the origin and the next origin coincide with each other. The same sentence setting process is performed in which the document data is set as the character attribute data indicating that matching characters are characters in the same sentence .
In the information processing apparatus of the present invention, the font collating unit calculates (1) the size of the bounding box of the font in the font collation data corresponding to the character recognized by the character recognizing unit, Estimating the character size of the recognized character; (2) when the plurality of fonts or the plurality of character sizes in the font collation data are estimated, the recognized character, the plurality of fonts, and the plurality of characters Image similarity is calculated by comparing either or both of the character sizes, and (3) the font and the character size having the highest similarity are selected, and the selected font and the character size are selected. When the similarity with respect to the recognized character is equal to or greater than a predetermined threshold, it is determined that the font and the character size are matched, and Similarity score is equal to or determines that said font and the character size corresponding to when the predetermined lower than the threshold value has not been collated without.
In the information processing apparatus according to the present invention, when it is determined by the font collating means that the corresponding font and the character size are not collated, the area of the image data recognized as the character is included in the document data. Character image switching means for deleting the character information and replacing it with line drawing data is further provided.
The information processing system of the present invention includes an image data input in the image forming apparatus, an information processing system including a server for changing the image data to the document data, the server, and the image data, a plurality of fonts and storage means for storing the font of font matching data character size, a character recognizing section for recognizing characters from the image data stored in the storage means, the image that has been recognized as the characters by the character recognition means the region of the data, recognized the character, and font matching means for matching or corresponding to any of a plurality of fonts and a plurality of character sizes of the font within the collation data stored in said storage means, said font verification unit wherein the document data in collated the font and the character size was as character attribute data And configure, and the drawing position location of the character and a character attribute data setting means for setting to the document data as the character attribute data so as to correspond to the image data, the character attribute data setting means (1) The font and the character size in the collated font collation data are compared with the image of the character of the document data to calculate the best matching coordinates, and based on the coordinates, Performing origin calculation processing for calculating the origin of the character; (2) specifying the font and the character size in the collated font collation data, and at least a bounding area that is a drawing area of the character data in the text data A box, a next origin which is the origin of the character and the origin of the next character calculated in the origin calculation process, a side bearing of the character, and the sentence A character drawing position correction setting process for setting the drawing data including the character width of the character data as the character attribute data for the document data. (3) For the adjacent characters in the document data, the origin and the next origin Is set to the document data as the character attribute data indicating that the adjacent character is a character in the same sentence. It is characterized by performing processing .

  According to the present invention, it is possible to provide an information processing apparatus capable of creating document data similar to the original document by setting the collated font and character size in the document data and setting the character drawing position. it can.

1 is a block diagram illustrating a configuration of an image forming system according to an embodiment of the present invention. It is a block diagram which shows the whole structure of the image forming apparatus shown in FIG. FIG. 2 is a schematic diagram of the image forming apparatus illustrated in FIG. 1. It is the schematic of the server shown in FIG. It is a block diagram which shows the control structure of the server shown in FIG. It is a flowchart of the image file formation process which concerns on embodiment of this invention. It is a conceptual diagram of the character recognition process shown in FIG. FIG. 7 is a conceptual diagram of character size estimation processing and matching font calculation processing shown in FIG. 6. FIG. 7 is a conceptual diagram of an origin calculation process and a character drawing position correction process shown in FIG. 6. It is a conceptual diagram of the drawing data which concern on the character drawing position correction process shown in FIG. It is a conceptual diagram of the same sentence setting process shown in FIG.

<Embodiment>
[Configuration of image forming system X]

First, the configuration of the image forming system X will be described with reference to FIG. The image forming system X includes, for example, an image forming apparatus 1 and a server 2.
The image forming apparatus 1 is an MFP or the like equipped with a scanner or the like, and transmits image data 200 (FIG. 5) to the server 2.
The server 2 is an information processing apparatus such as a PC server such as a PC / AT compatible machine or an ARM (registered trademark) server installed in a data center or the like. The server 2 is a terminal such as a user's PC (Personal Computer), a smartphone, a mobile phone, or a PDA (Personal Digital Assistant), and is configured to execute an installed server program and function as an information processing apparatus. Also good. The server 2 performs image file conversion processing described later on the image data acquired from the image forming apparatus 1.
The image forming apparatus 1 and the server 2 are connected to a network 5 such as a LAN (Local Area Network), a wireless LAN, a WAN (Wide Area Network), and a mobile phone network.

[Entire Configuration of Image Forming Apparatus 1]
Next, the overall configuration of the image forming apparatus 1 will be described with reference to FIG. The image forming apparatus 1 includes an image processing unit 11, a document reading unit 12, a document feeding unit 13, a conveyance unit (a paper feed roller 42 b, a conveyance roller pair 44, a discharge roller pair 45), a network transmission / reception unit 15, and an operation panel unit 16. The image forming unit 17 (image forming unit), the storage unit 19 and the like are connected to the control unit 10. The operation of each unit is controlled by the control unit 10.

The control unit 10 includes a general purpose processor (GPP), a central processing unit (CPU), a micro processing unit (MPU), a digital signal processor (DSP), a graphics processing unit (GPU), and an application specific processor (ASIC). Information processing means such as a processor for a specific application)
The control unit 10 reads out a control program stored in the ROM or HDD of the storage unit 19, develops the control program in the RAM, and executes it to operate as each unit of a functional block described later. Further, the control unit 10 controls the entire apparatus in accordance with predetermined instruction information input from an external terminal (not shown) or the operation panel unit 16.

The image processing unit 11 is control arithmetic means such as a DSP (Digital Signal Processor) or a GPU (Graphics Processing Unit). The image processing unit 11 is a unit that performs predetermined image processing on various types of image data, and performs various types of image processing such as enlargement / reduction, density adjustment, gradation adjustment, and image improvement.
The image processing unit 11 stores the image read by the document reading unit 12 in the storage unit 19 as print data. At this time, the image processing unit 11 can also convert the print data into a file unit in a format such as PDF or TIFF.

The document reading unit 12 is a (scan) unit that reads a set document.
The document feeding unit 13 is a unit that transports a document read by the document reading unit 12.
The image forming unit 17 is a unit that forms an image on a recording sheet from data stored in the storage unit 19 and read by the document reading unit 12 or acquired from an external terminal according to a user output instruction.
The transport unit transports the recording paper from the paper feed cassette 42a (FIG. 3), causes the image forming unit 17 to form an image, and then transports the recording paper to the stack tray 50.
The operations of the document reading unit 12, the document feeding unit 13, the transport unit, and the image forming unit 17 will be described later.

The network transmission / reception unit 15 is a network connection unit including a LAN board, a wireless transceiver, and the like for connecting to an external network such as a LAN, a wireless LAN, a WAN, and a mobile phone network.
The network transmission / reception unit 15 transmits / receives data on a data communication line and transmits / receives voice signals on a voice telephone line.
The network transmission / reception unit 15 may be connected to an external terminal such as a PC (Personal Computer), a smartphone, a PDA (Personal Data Assistant), a mobile phone, or a server (not shown) via a network.

The operation panel unit 16 includes a display unit such as an LCD, a numeric keypad, a start button, a cancel button, a button for switching an operation mode such as copying, FAX transmission, and a scanner, and a job related to printing, transmission, storage, and recording of a selected document. And an input unit such as a button or a touch panel for giving an instruction related to the execution of.
The operation panel unit 16 acquires a user instruction to the image forming apparatus 1. In response to the user's instruction, an image filing process described later is started. It is also possible to input and change information of each user according to user instructions.
Further, the operation panel unit 16 may include a connection unit for connecting a recording medium storing the image data 200 (FIG. 5) such as a USB memory, a flash memory card, or an external device.

The storage unit 19 is a storage unit that uses a semiconductor memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory) or a recording medium such as an HDD (Hard Disk Drive).
Even if the RAM of the storage unit 19 is in a power saving state, the stored contents are held by a function such as self-refresh.
A control program for controlling the operation of the image forming apparatus 1 is stored in the ROM and HDD of the storage unit 19. In addition to this, the storage unit 19 also stores user account settings. The storage unit 19 may include a storage folder area 301 for each user.

In the image forming apparatus 1, the control unit 10 and the image processing unit 11 may be integrally formed, such as a CPU with a built-in GPU or a chip-on-module package.
The control unit 10 and the image processing unit 11 may include a RAM, a ROM, a flash memory, and the like.
Further, the image forming apparatus 1 may include a FAX transmission / reception unit that performs facsimile transmission / reception.

[Operation of Image Forming Apparatus 1]
Next, the operation of the image forming apparatus 1 according to the embodiment of the present invention will be described with reference to FIG.
The document reading unit 12 is arranged on the upper part of the main body unit 14, and the document feeding unit 13 is arranged on the upper side of the document reading unit 12. The stack tray 50 is disposed on the recording paper discharge port 41 side formed in the main body unit 14, and the operation panel unit 16 is disposed on the front side of the image forming apparatus 1.

The document reading unit 12 includes a scanner 12a, a platen glass 12b, and a document reading slit 12c. The scanner 12 a includes an exposure lamp, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) imaging sensor, and the like, and is configured to be movable in the document transport direction by the document feeder 13.
The platen glass 12b is an original table made of a transparent member such as glass. The document reading slit 12 c has a slit formed in a direction orthogonal to the document transport direction by the document feeding unit 13.

When reading a document placed on the platen glass 12b, the scanner 12a is moved to a position facing the platen glass 12b, and reads the document while scanning the document placed on the platen glass 12b, thereby obtaining image data 200. And the acquired image data 200 is output to the control unit 10 (FIG. 2) provided in the main body unit 14.
When reading the document conveyed by the document feeding unit 13, the scanner 12a is moved to a position facing the document reading slit 12c, and the document feeding unit 13 conveys the document through the document reading slit 12c. The document is read in synchronization with the operation to acquire the image data 200, and the acquired image data 200 is output to the control unit 10 provided in the main body unit 14.

The document feeding unit 13 includes a document placement unit 13a, a document discharge unit 13b, and a document transport mechanism 13c. The originals placed on the original placement unit 13a are sequentially fed out one by one by the original conveyance mechanism 13c, conveyed to a position facing the original reading slit 12c, and then discharged to the original discharge unit 13b.
The document feeding unit 13 is configured to be retractable, and the upper surface of the platen glass 12b can be opened by lifting the document feeding unit 13 upward.

The main body unit 14 includes the image forming unit 17, and includes a paper feeding unit 42, a paper conveyance path 43, a conveyance roller pair 44, and a discharge roller pair 45. The paper feed unit 42 includes a plurality of paper feed cassettes 42 a that store recording papers of different sizes or orientations, and a paper feed roller 42 b that feeds the recording papers one by one from the paper feed cassette 42 a to the paper transport path 43. Yes. The paper feed roller 42b, the transport roller pair 44, and the discharge roller pair 45 function as a transport unit. The recording paper is conveyed by this conveyance unit.
The recording paper fed to the paper transport path 43 by the paper feed roller 42 b is transported to the image forming unit 17 by the transport roller pair 44. Then, the recording paper on which recording is performed by the image forming unit 17 is discharged to the stack tray 50 by the discharge roller pair 45.

  The image forming unit 17 includes a photosensitive drum 17a, an exposure unit 17b, a developing unit 17c, a transfer unit 17d, and a fixing unit 17e. The exposure unit 17b is an optical unit including a laser device, a mirror, a lens, an LED array, and the like, and outputs light or the like based on image data to a photosensitive drum 17a that is primarily charged by a charging unit (not shown). And an electrostatic latent image is formed on the surface of the photosensitive drum 17a. The developing unit 17c is a developing unit that develops the electrostatic latent image formed on the photosensitive drum 17a using toner, and forms a toner image based on the electrostatic latent image on the photosensitive drum 17a. The transfer unit 17d transfers the toner image formed on the photosensitive drum 17a by the developing unit 17c to a recording sheet. The fixing unit 17e heats the recording paper on which the toner image is transferred by the transfer unit 17d to fix the toner image on the recording paper.

[Entire configuration of server 2]
Next, the overall configuration of the server 2 (information processing apparatus) will be described with reference to FIG.
The server 2 includes a control unit 20, a network transmission / reception unit 25, and a storage unit 29. The operation of each unit is controlled by the control unit 20.

  The control unit 20 is an information processing unit having calculation / control capabilities such as GPP, CPU, MPU, DSP, GPU (Graphics Processing Unit), ASIC, and the like. The control unit 20 executes the program stored in the storage unit 29 using hardware resources.

The network transmitting / receiving unit 25 is a standard LAN interface such as 1000BASE-T / 100BASE-TX, a wireless LAN interface, a portable radio wave transmitting / receiving device, or the like for connecting to the network 5. The server 2 can transmit / receive various data via the network transmitting / receiving unit 25.
The network transmitting / receiving unit 25 may be connected to an external hub, router, or the like and connected to the network 5.

  The storage unit 29 is a storage means including a RAM, a flash memory disk such as an SSD (Solid State Drive), an HDD (Hard Disk Drive), a magnetic tape device, an optical disk device, and the like. The storage unit 29 stores an OS (Operating System) for causing the server 2 to function and various programs and data of the server. Details of these programs and data will be described later.

[Control configuration of server 2]
Here, the control configuration of the server 2 will be described with reference to FIG.
The control unit 20 of the server 2 includes an image data acquisition unit 100 (image data acquisition unit), a character recognition unit 110 (character recognition unit), a font collation unit 120 (font collation unit), and a character attribute data setting unit 130 (character attribute data). Setting means) and a character image switching unit 140 (character image switching means).
The storage unit 29 stores image data 200, document data 210, and font collation data 220.
The network transmission / reception unit 25 receives the image data 200 and transmits the converted document data 210.

The image data acquisition unit 100 acquires image data 200 for performing OCR (Optical Character Recognition) or the like.
The image data acquisition unit 100 acquires the image data 200 transmitted from the image forming apparatus 1 and received by the network transmission / reception unit 25 via the network 5 and stores it in the storage unit 29.
At this time, the image data acquisition unit 100 may cause the document reading unit 12 to scan a document including printed characters and transmit the image data 200 to the server 2 according to a user's instruction such as “network scan”.
The image data acquisition unit 100 stores the image data 200 scanned by the document reading unit 12 or the image data 200 in the document box in accordance with a user instruction that can operate the operation panel unit 16 of the image forming apparatus 1. 2 may be transmitted.
Further, the image data acquisition unit 100 may acquire the image data 200 from a connected recording medium, an external terminal, a server, or the like.

The character recognition unit 110 recognizes characters by optical character recognition or the like of the image data 200 acquired by the image data acquisition unit 100.
For example, the character recognition unit 110 recognizes a region 301 in which a character is drawn from the image data 200, calculates a line drawing vector of the character, and the like, and determines a character code by using an artificial neural network or the like. Then, the character recognition unit 110 sets the character code determined as the character attribute data 211 in the document data 210.
In addition, the character recognition unit 110 also stores in the storage unit 29 data such as the coordinates of the area 301 recognized as a character in the image data 200 and the vector of the line drawing of the character during optical character recognition, for example. Also good.

The font collation unit 120 corresponds to any of a plurality of fonts and a plurality of character sizes included in the font collation data 220 for the region 301 of the image data 200 recognized as a character by the character recognition unit 110. Or collate.
For example, the font collation unit 120 calculates the size of the “bounding box” that is the drawing area of the drawing data of the font of the font collation data 220 corresponding to the character code output by the character recognition unit 110, and sets the character size. presume. When a plurality of fonts or character sizes are estimated, the font collation unit 120 performs image collation by comparing one or both of the fonts and character sizes and calculating a similarity. The font collation unit 120 selects a font and a character size having the highest similarity. If the selected font and character size have a similarity equal to or greater than a predetermined threshold, the font collation unit 120 determines that the font and character size correspond to each other and have been collated. Further, when the similarity is lower than a predetermined threshold, the font collation unit 120 determines that there is no corresponding font and character size and the collation has not been performed.

The character attribute data setting unit 130 sets the font and character size collated by the font collation unit 120 in the document data 210, and sets the character drawing position so as to correspond to the image data 200.
For example, the character attribute data setting unit 130 sets, for each character of the character attribute data 211 of the document data 210, a font and a character size determined to be applicable by the font collation unit 120. Also, the character attribute data setting unit 130 sets the drawing position corresponding to the image data 200 when the character is browsed or printed. Therefore, the character attribute data setting unit 130 sets the drawing origin 401 (FIG. 8), the character width 404, and the left side bearing 403 to correspond to the image data 200 as the character drawing position. Note that the character attribute data setting unit 130 may also set the document data 210 in which the side bearings such as right, upper, and lower are set so as to correspond to the image data 200.
In addition, the character attribute data setting unit 130 calculates a next origin 402 where the next character is drawn corresponding to the origin 401 calculated corresponding to the character collated by the font collation unit 120, and the next origin 402 And the origin 401 of the next character coincide with each other, it is set in the document data 210 as a character in the same sentence 321 (FIG. 11).
Thereby, when the document data 210 is browsed or output by the image forming unit 17, characters are drawn at substantially the same position as the image data. In addition, characters can be grouped for each sentence similar to the original document from which the image data 200 is based, and the convenience for the user is enhanced.

The character image switching unit 140 causes the font collation unit 120 to select any of a plurality of fonts and a plurality of character sizes included in the font collation data 220 with respect to the drawing position of the document data 210 corresponding to the region 301 of the image data 200. If the image data is not matched, the drawing data of the line drawing or image that does not include characters is set in the document data 210.
Note that the character image switching unit 140, when the font collation unit 120 collates with one of a plurality of fonts and a plurality of character sizes included in the font collation data 220, draws the recognized character drawing data. The character attribute data 211 may be left set in the document data 210.

The image data 200 is data including a character image having a resolution that enables optical character recognition. The image data 200 may be, for example, data read by the document reading unit 12 and image processed by the image processing unit 11.
Further, the image data 200 may be a RGB image, a CMYK color, a monochrome or grayscale bitmap image file, a file that is lightly compressed with a run length, LZW, or the like.
Further, the image data 200 is already transferred to PDF, JPG (Joint Photographic Experts Group), PNG (Portable Network Graphics), BMP (Bitmap Image) by the control unit 20 of the server 2, the control unit 10 of the image forming apparatus 1, and the image processing unit 11. ) Or TIFF (Tagged Image File Format) or the like.
The image data 200 may be an image file such as JPG or PDF including an image acquired from a document box in the storage unit 29, a connected recording medium, an external terminal, another server, or the like.
In addition, when the image data 200 is a PDF, character attribute data 211 that has already been recognized may be included.
The image data 200 may be image data such as JBIG received by facsimile.

The document data 210 is data for a document including characters.
The document data 210 may be, for example, an electronic document such as PDF, PS (PostScript (registered trademark)), PDL (Page Description Language), or a file having a format close to print data. The document data 210 may be a hypertext file such as HTML (Hyper Text Markup Language). The document data 210 may be a file of various application software (hereinafter referred to as “application”) such as a word processor, spreadsheet software, and drawing software. In this case, the document data 210 includes, for example, “.docx (Microsoft (registered trademark) Word (registered trademark))”, “.xlsx (Microsoft (registered trademark) Excel (registered trademark))”, SVG (Scalable Vector Graphics). Or other files. The document data 210 may be a page file for printing.
In the case of PDF, the document data 210 may include the character attribute data 211 and the image data 200, or may include only the character attribute data 211.
In the present embodiment, an example in which the document data 210 is PDF or PS will be described.

The character attribute data 211 is character code and character attribute data included in the document data 210.
When the image data 200 is recognized by the character recognition unit 110 and the document data is created, the character attribute data 211 may include only the character code and data of a rough character drawing position.
In the character attribute data 211, data such as the font and character size of each character and the character drawing position may be set by the character attribute data setting unit 130. In the character attribute data 211, a drawing origin 401 (FIG. 8), a next origin 402, a left side bearing 403, a character width 404, and the like may be set as data of a character drawing position. At this time, a side bearing other than the left side bearing 403 may be set. In addition, the character attribute data 211 may be set as text data such as text 321 (FIG. 11), which is a group of characters, as a bounding box, font box, or the like.
Further, the character attribute data 211 may include font dictionary data object data when the document data 210 is PS, PDF, PDL, or the like.

The font collation data 220 is data for collating the image data 200 when the character code is drawn using the character code as a search key.
For example, the font collation data 220 may include, as a character table or the like, bitmap data of each character of a plurality of fonts created in increments of 0.1 points, for example. Each font of the font collation data 220 may include a standard PC, an in-house device of the image forming apparatus 1, a standard font for printed matter, and the like. Further, as the character size of the font collation data 220, for example, data having a size of 8 points to 96 points may be included corresponding to the dpi (dot per inch) of the image data 200 or the like. In addition, the font collation data 220 may include data when character modification such as bold, italic, underline, and shading is performed for each character size of each font. Further, the font collation data 220 may include the coordinates of the bounding box in which the character is drawn for each character in the character table for each font and each character size.
The font collation data 220 may be vector (line drawing) data of a plurality of fonts instead of bitmap data. In this case, for the area of the image data 200, each bit map with a changed character size may be drawn for comparison.

Here, the control unit 20 of the server 2 executes the control program stored in the storage unit 29, whereby the image data acquisition unit 100, the character recognition unit 110, the font collation unit 120, the character attribute data setting unit 130, the character It is made to function as the image switching unit 140.
Moreover, each part of the above-mentioned server 2 becomes a hardware resource which executes the program of this invention.

[Image File Processing by Image Forming Apparatus 1]
Next, an image filing process by the image forming apparatus 1 according to the embodiment of the present invention will be described with reference to FIGS.
In the image filing process of this embodiment, first, image data 200 is acquired, optical character recognition is performed, and document data 210 is created. Based on the character code output by the optical character recognition, it is verified whether the region 301 recognized as the character of the image data 200 is output with which character size of which font. For character portions that can be collated, the character drawing position of the document data 210 is corrected, and the same character information as that of the original document of the image data 200 is written. At this time, the origin of drawing, character width, and side bearing are set. For portions that could not be collated, the characters in the document data 210 are deleted and replaced with vector drawing data.
In the image filing process of the present embodiment, the control unit 20 mainly executes a program stored in the storage unit 29 using hardware resources in cooperation with each unit.
Hereinafter, the details of the image filing process will be described step by step with reference to the flowchart of FIG.

(Step S100)
First, the control unit 20 performs an image data acquisition process as the image data acquisition unit 100.
Further, the control unit 20 acquires the image data 200 from the image forming apparatus 1 or the like via the network transmission / reception unit 25 and stores it in the storage unit 29.
The control unit 20 also creates a document data 210 file corresponding to the image data 200. At this point, the text included in the image data 200 may be copied to the document data 210, although no characters have been set yet.
Note that the control unit 20 may transmit a remote scan command to the image forming apparatus 1 via the network 5 and cause the document reading unit 12 to scan a document on which characters are printed. In this case, the control unit 10 of the image forming apparatus 1 transmits the scanned image data 200 to the server 2. At this time, the control unit 10 may perform processing such as deforming an end portion of a spread document such as a book. In addition, the control unit 10 may remove dust from the image, remove moire, increase the sharpness of the image, or perform processing for DPI conversion. Further, the control unit 10 may perform these image processes at a high speed using the GPU of the image processing unit 11 or the like.
Further, the control unit 20 may acquire the image data 200 from a recording medium (not shown), an external terminal, a server, or the like and store the image data 200 in the storage unit 29.
Further, the control unit 20 may acquire the image data 200 received by a FAX transmission / reception unit (not shown) of the image forming apparatus 1 and store it in the storage unit 29.
The control unit 20 may convert the acquired image data 200 into data of a specific format.

(Step S101)
Next, the control unit 20 performs character recognition processing as the character recognition unit 110.
According to FIG. 7A, the control unit 20 searches the image data 200 for a portion where a character is drawn, and performs character recognition for each character in the searched portion. The control unit 20 includes data such as the character drawing area 311 of the recognized character in the character attribute data 211 and sets it in the document data 210.
According to FIG. 7B, a drawing position is set in the character drawing area 311 so that the character drawing area 311 is roughly viewed or printed at the same position as the image data 200 in the document data 210. For example, when the document data 210 is PS, PDF, PDL or the like, the character drawing area 311 may be defined as a collection of font boxes 410 including the origin of the character drawing area 311 and character codes in an encoding format such as UTF8. Good.
In the character drawing area 311 at this time, only the origin 400 of each font box 410 and a default font may be set. Therefore, when drawing data in the character drawing area 311 at this time is drawn, a drawing result different from that of the original document from which the image data 200 is based is obtained.

The control unit 20 may store the recognized character code and the coordinates of the area 301 of the image data 200 recognized as a character in the storage unit 29 as temporary data. Note that the control unit 20 may also store character vector data calculated during character recognition in the storage unit 29 as temporary data.
Moreover, the control part 20 may recognize figures other than a character. If the image data 200 includes character attribute data 211 that has already been recognized, this character recognition process can be skipped.

(Step S102)
Next, the control unit 20 performs a character size estimation process as the font collation unit 120.
Referring to FIG. 8A, in the matched font calculation process from this process to step S104, the control unit 20 calculates the font type and the character size for each OCR character. The control unit 20 also includes a bounding box 411 that is a drawing area for each character, a font drawing origin 401, a next origin 402 that is the next character origin, a left side bearing 403 that is a left margin in character drawing, a character Drawing data such as width 404 is set. At this time, the control unit 20 calculates the origin 401, the character width 404, and the left side bearing 403 so that the coordinates coincide with the image data 200. For this reason, first, in this process, the control unit 20 estimates the character size.
According to FIG. 8B, the control unit 20 includes the size of each character area 302 of the area 301 recognized as the character of the image data 200 calculated by the character recognition process described above in the font collation data 220, and The character size is estimated by comparing the size of the bounding box of each font at each character size. In this comparison, the control unit 20 compares the size of the region 302 in consideration of the size of coordinates where characters are actually drawn corresponding to the bounding box 411. At this time, the control unit 20 also compares the bounding box of the data when the character modification is performed. In addition, the control unit 20 may refer to information such as the size, dpi, and resolution of the image data 200 itself in the comparison.
The control unit 20 estimates that the size of the bounding box matches by the comparison as the font type and the character size. At this time, a plurality of fonts and a plurality of character sizes may be estimated. The control unit 20 temporarily stores the estimated font type and character size in the storage unit 29.
When searching, the control unit 20 may compare the FontBBox information of the character drawing area 311 calculated by the character recognition process described above with reference to the character size.

(Step S103)
Next, the control unit 20 determines whether a plurality of fonts and a plurality of character sizes are estimated as the font collation unit 120. The control unit 20 determines Yes when a plurality of font types or character sizes are estimated in the character size estimation process described above. In other cases, that is, if only one font type and character size is estimated or matched, the control unit 20 determines No.
In the case of Yes, the control part 20 advances a process to step S104.
In No, the control part 20 advances a process to step S105.

(Step S104)
When a plurality of font types or character sizes are estimated, the control unit 20 performs matching font calculation processing as the font collation unit 120.
According to FIG. 8C, the control unit 20 compares each bitmap image of a plurality of font types or character sizes with an actual character image in the region 302, and finds the best match. Matches to search. For this comparison, for example, the control unit 20 may set windows such as 8 to 192 dots in the upper, lower, left, and right directions, and may calculate the lightness similarity of the dark spots of each pixel using dynamic programming or the like. At this time, the control unit 20 calculates the similarity considering the printing conditions such that the outline of the character tends to be thick in the printed document, or changes to black and white binary and uses bit addition or the like for high speed. The degree of similarity may be calculated. For example, the control unit 20 may select the image having the highest similarity as the closest image. Further, the control unit 20 may calculate coordinates on the window at this time.
The control unit 20 temporarily stores the font of the selected closest image, coordinates on the window, similarity, and the like in the storage unit 29.
Thus, only when a plurality of font types or character sizes are calculated, the font and character size can be searched at high speed by comparing whether or not the characters match.

(Step S105)
Here, the control unit 20 determines whether the collation is successful as the font collation unit 120. For example, when only one font type and character size matching is estimated in the character size estimation process described above, or when the similarity is equal to or greater than a predetermined threshold value in the matching font calculation process. If there is, it is determined that the recognized character corresponds to one of the fonts and the character size in the font collation data 220, that is, the collation is successful, and the decision is Yes. In other cases, for example, when the character size estimation process does not calculate a matching font type and character size, or the matching font calculation process indicates that the similarity is less than a predetermined threshold. In such a case, it is determined that the corresponding font and character size were not found, that is, the collation failed, No.
In the case of Yes, the control part 20 advances a process to step S106.
In No, the control part 20 advances a process to step S110.

(Step S106)
When the collation is successful, the control unit 20 performs origin calculation processing as the character attribute data setting unit 130.
According to FIG. 9, the control unit 20 compares the actual character image in the region 302 using the estimated font and character size, calculates the best matching coordinate, and based on this coordinate The origin 401 is calculated. For this comparison, for example, a window such as 8 to 192 dots in the vertical and horizontal directions is set, and by using dynamic programming or the like, the similarity of the lightness of the dark spot of each pixel is calculated, and the most coincident coordinate is calculated. Also good. At this time, the control unit 20 calculates the degree of similarity in consideration of the printing conditions, or calculates the degree of similarity at high speed using bit addition or the like after changing to black and white binary, as in the above-described matching font calculation process. Or you may. At this time, the control unit 20 may also calculate the next origin point 402 (FIG. 8) based on the origin point 401.
Note that the control unit 20 may calculate the drawing position from, for example, the coordinates on the above-described collation processing window and the coordinates of the area 301 recognized as the character of the image data 200. In addition, when the character is drawn, the control unit 20 may calculate the degree of similarity by checking whether or not the image actually overlaps the image in the region 301 of the image data 200.

(Step S107)
Next, the control unit 20 performs a character drawing position correction setting process as the character attribute data setting unit 130.
The control unit 20 specifies the origin 401 calculated in the above-described origin calculation process, specifies the font and character size of the character size estimation process or the matching font calculation process, and sets the character attribute data 211 in the document data 210 To do.
At this time, for example, the control unit 20 changes the font and character size of the document data 210 and the drawing position of the font. When the document data 210 is PS, PDF, PDL, or the like, the control unit 20 registers a font dictionary having a font and a character size with a similarity equal to or greater than the threshold as the character attribute data 211, and sets the drawing position to “write”. Set as "Position" information. Here, the control unit 20 sets the drawing position of characters in the character attribute data 211 of the document data 210 so as to correspond to the accurate drawing position on the calculated image data 200. For example, the control unit 20 includes a bounding box 411, a font drawing origin 401, a next origin 402 that is the origin of the next character, a left side bearing 403, a character width 404, and the like as shown in FIG. Drawing data is set in the character attribute data 211 of the document data 210.

FIG. 10 shows an example in which this drawing data is registered in the font dictionary of the character attribute data 211 of the document data 210 that is PS.
The drawing data 501 is an example in which “type 3” is designated as a font and a glyph coordinate system and a font boundary are designated.
The drawing data 502 is an example of an array definition for converting a character code into a character name. In this example, the code “80” is converted to “P”.
The drawing data 503 is an example of a dictionary for storing glyph descriptions (vectorized characters).
The drawing data 504 is an example of a code indicating designation of a glyph creation procedure. This code indicates character boundaries, dictionary acquisition, and the like.
The drawing data 505 is an example of a code indicating font dictionary acquisition, font size designation, position designation, character display, and the like necessary for character drawing.

(Step S108)
Next, the control unit 20 determines whether there is a character whose origin 401 and the next origin 402 match as the character attribute data setting unit 130. When there are a plurality of characters in the character attribute data 211 set in the document data 210, the control unit 20 determines whether the distance between the origin 401 and the next origin 402 is equal to or less than a predetermined threshold for adjacent characters. Is determined. The control part 20 determines with Yes, when there exists such a character pair below a threshold value. That is, the control unit 20 determines Yes when the next origin 402 calculated corresponding to the origin 401 calculated corresponding to the collated character matches the origin 401 of the next character. In the example of FIG. 11, the control unit 20 determines Yes because the next origin 402 of the character preceding “gakpylp” and the origin 401 of the next character match. In other cases, the control unit 20 determines No.
In the case of Yes, the control part 20 advances a process to step S109.
In No, the control part 20 complete | finishes an image file formation process.

(Step S109)
When there is an adjacent character in the character attribute data 211 where the origin 401 and the next origin 402 match, the control unit 20 performs the same sentence setting process as the character attribute data setting unit 130.
According to FIG. 11, when the next origin 402 of the previous character matches the origin 401 of the next character, there is a high possibility of a character in the same sentence of the original document from which the image data 200 is based. Therefore, in the example of FIG. 11, the control unit 20 sets the characters as characters in the same sentence 321 in the character attribute data 211.
Thereafter, the control unit 20 ends the image file processing.

(Step S110)
When the font and the character size are not successfully verified for the area recognized as the character of the image data 200, the control unit 20 performs a character image switching process as the character image switching unit 140.
The control unit 20 deletes the character information in the document data 210 and replaces it with vector (line drawing) data for the region 301 of the image data 200 recognized as a character. As the replacement vector data, the character line drawing vector data calculated by the above-described character recognition processing may be used. Further, the control unit 20 may output the image of the area 301 to the document data 210 as it is.
Thus, the image filing process according to the embodiment of the present invention is completed.

With the configuration described above, the following effects can be obtained.
A conventional character recognition apparatus cannot accurately set the font and size of characters when converting image data into document data. For this reason, the image data cannot be converted into document data similar to the original document.
On the other hand, the server 2 according to the embodiment of the present invention is an information processing apparatus that can change the image data 200 to the document data 210, the image data acquisition unit 100 that acquires the image data 200, and the image data acquisition The character recognition unit 110 that recognizes characters from the image data 200 acquired by the unit 100 and the region 301 of the image data 200 recognized as characters by the character recognition unit 110 include a plurality of fonts and a plurality of characters. Font collation unit 120 for collating which size corresponds, font and character size collated by font collation unit 120 are set in document data 210, and drawing origin 401, character width 404, And the character attribute data setting unit 1 for setting the left side bearing 403 to correspond to the image data 200 Characterized in that it comprises a 0 and.
With this configuration, it is possible to restore the format of characters printed on the original document from which the image data 200 is based. As a result, it is possible to convert the document data 210 that has been character-recognized from the image data 200 into a file similar to the files of various applications. That is, it can be converted into document data 210 similar to the original document.
When the converted document data 210 is read by the corresponding application, the character font type, the character size, the character modification information such as bold / italic, the character drawing position, etc. can be almost surely reproduced. It becomes. For this reason, the font information of the document data 210 can be reproduced, and characters can be easily corrected by various applications. Thereby, the convenience of the user is increased.
In particular, by setting the drawing origin 401, the character width 404, and the left side bearing 403, the character position can be easily set in the document data 210, and the drawing data structure can be simplified. For this reason, the versatility of the document data 210 can be improved.

Further, in the technique such as Patent Document 1, since the document data 210 includes data drawn in vectors, there is a problem that the read application may not be able to correct characters. If the character portion of the image data 200 is not in the file as character information, the original document cannot be restored.
On the other hand, the server 2 according to the present embodiment collates with the image table set for all fonts and character sizes included in the font collation data 220 with the character code detected by the character recognition unit 110. It is possible to almost certainly extract the font and character size.
Further, the detected character image is compared with the image data 200, and the character drawing position set in the document data 210 by the character recognition means is corrected to an accurate value.
With this configuration, it is possible to almost certainly reproduce the original document. Further, the converted document data 210 can be printed so as to be in the same state as the original document. At this time, since the character attribute is set in the character attribute data 211 in the document data 210, blurring and blurring resulting from scanning are reduced, and high-quality printing can be performed.

In the server 2 according to the embodiment of the present invention, the character attribute data setting unit 130 causes the character to be drawn next corresponding to the origin 401 calculated corresponding to the character collated by the font collation unit 120. The next origin 402 is calculated, and when the next origin 402 and the origin 401 of the next character coincide with each other, it is set in the document data 210 as a character in the same sentence 321. In other words, if the distance between the next origin 402 of the previous character and the origin 401 of the next character is equal to or less than a predetermined value, it is set in the character attribute data 211 of the document data 210 that it belongs to the same sentence 321.
With this configuration, even when a character is recognized as a part of a separate sentence by OCR or the like, it can be recombined as a unified sentence. For this reason, it becomes easy to extract a sentence from the document data 210, and user convenience is improved.

The server 2 according to the embodiment of the present invention uses the font collation unit 120 to select one of a plurality of fonts and a plurality of character sizes with respect to the drawing position of the document data 210 corresponding to the area 301 of the image data 200. The character image switching unit 140 sets line drawing data of recognized characters when collated as appropriate, and sets line data or image drawing data not including characters when not collated. To do.
By comprising in this way, it becomes possible to detect the location misrecognized in character recognition. Thus, by deleting the misrecognized character attribute data 211 included in the converted document data 210, the user can easily edit the document data 210, and the user's convenience can be improved. Further, for example, the document data 210 in a complete state can be created by deleting characters in a portion where character recognition has failed, such as a company logo.
Further, even when the collation fails for the font and the character size, the document data 210 capable of high-quality printing can be obtained by setting vector drawing data for the character portion. Further, even when character recognition fails in the middle of a sentence, it is possible to reduce a sense of discomfort when browsing or printing the part.

In the present embodiment, when the document data 210 is PDF, information about the font is set with the collated font and character size for the bounding box of the font created by character recognition.
With this configuration, information such as the font and writing position of the character attribute data 211 included in the PDF becomes accurate, and character information corresponding to the style can be easily extracted from the PDF. In addition, conversion from PDF to a file of an application such as a word processor or spreadsheet software becomes easy.

In the description of the font collation process of the above-described embodiment, collation is performed after estimating the character size.
However, the control unit 20 may search the character table of the font collation data 220 by brute force without estimating the character size first. Further, when only vector data is stored as the font collation data 220, when the character is rendered as primary data, the ratio of the number of pixels that match the region 301 of the image data 200 is calculated, and collation is performed. You may go. Moreover, the control part 20 may perform collation at high speed using a GPU calculating function etc. Further, the control unit 20 may use data such as a character line drawing vector created by the above character recognition processing at the time of collation.
With this configuration, it is possible to reduce the storage capacity of the storage unit 29 or speed up the collation.

In the above-described embodiment, an example of conversion to PDF or PS has been described. However, the present invention is not limited to this, and conversion to various document formats is possible.
In addition, when the image file is converted into a word processor document file, a spreadsheet software file, an HTML file, or the like, the control unit 20 calculates information on the drawing position of the corrected character in units of pages, and calculates the standard font and line feed. Also, settings such as character spacing, margins, header / footer, etc. may be calculated and set in the document data 210. As a result, the same page settings as the original file can be restored, the document file can be easily modified, and the convenience for the user can be improved.

Further, in the above-described embodiment, it is described that the corresponding character attribute data 211 is deleted from the document data 210 when collation fails, but it may be left.
In addition, the server 2 performs character recognition again by designating the same font or character size as the surrounding collated characters for the characters that have not been collated by the font collation unit 120 with any of the plurality of fonts and the plurality of character sizes. You may provide the character re-recognition part. With this configuration, the accuracy of character recognition can be increased. In addition, it is possible to reliably recognize a character even in a portion where recognition is likely to fail in the conventional character recognition method, such as an underlined character.
Further, in the above-described embodiment, the character recognition unit 110 is described to output the character attribute data 211 to the document data 210 first, but the present invention is not limited to this. That is, the character recognition unit 110 may output a character code or the like as temporary data to the storage unit 29, and the character attribute data setting unit 130 may set the character code or the like in the document data.
With this configuration, the processing can be simplified, the correction of the document data 210 can be reduced, and the processing can be speeded up.

Further, the image filing process of the present embodiment may be executed by the image forming apparatus 1. Of the image filing processing, processing that requires computing performance, such as character recognition processing and collation processing, is executed by the server 2, and other processing is executed by the image forming apparatus 1. Also good. Further, such processing may be executed by a server 2 on a so-called “cloud” having a large number of GPUs or the like, instructed from a user terminal.
With this configuration, the costs of the control unit 10, the image processing unit 11, and the storage unit 19 of the image forming apparatus 1 can be reduced. Further, the processing can be completed in a short time, and power consumption can be reduced.
Further, the image filing process of the present embodiment can be applied to a system that does not use the image forming apparatus 1. In other words, a configuration may be used in which the network scanner and the server 2 to which the scanner is separately connected by USB or the like are used as the information processing apparatus.

  Further, the configuration and operation of the above-described embodiment are examples, and it goes without saying that they can be appropriately modified and executed without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 Image forming apparatus 2 Server 5 Network 10, 20 Control part 11 Image processing part 12 Original reading part 12a Scanner 12b Platen glass 12c Original reading slit 13 Original feeding part 13a Original placing part 13b Original discharge part 13c Original conveyance mechanism 14 Main body Sections 15 and 25 Network transmission / reception section 16 Operation panel section 17 Image forming section 17a Photosensitive drum 17b Exposure section 17c Development section 17d Transfer section 17e Fixing sections 19 and 29 Storage section 41 Discharge port 42 Paper feed section 42a Paper feed cassette 42b Paper feed Roller 43 Paper transport path 44 Transport roller pair 45 Discharge roller pair 50 Stack tray 100 Image data acquisition unit 110 Character recognition unit 120 Font collation unit 130 Character attribute data setting unit 140 Character image switching unit 200 Image data 210 Document data 211 Character attribute data 22 Font verification data 301, 302 area 311 character drawing region 321 sentences 400, 401 home 402 primary origin 403 left side bearing 404 character width 410 Font box 411 bounding box 420 characters 501,502,503,504,505 drawing data X imaging system

Claims (4)

The acquired image data, the information processing apparatus for outputting change in the document data,
Storage means for storing the image data and font collation data of a plurality of fonts and fonts of character sizes;
Character recognition means for recognizing characters from the image data stored in the storage means ;
The area of the image data is recognized as the character by the character recognition means, recognized the character, fall into one of a plurality of fonts and a plurality of character sizes of the font within the collation data stored in said storage means Font matching means for checking or matching,
The font set for the document data to the font and the character size is verified by collating means as character attribute data, and said drawing position location of the character as the character attribute data so as to correspond to the image data Character attribute data setting means for setting the document data ,
The character attribute data setting means includes:
(1) The font and the character size in the collated font collation data are compared with the image of the character of the document data to calculate the best matching coordinate, and the character is based on the coordinate. Perform the origin calculation process to calculate the origin of
(2) Designating the font and the character size in the collated font collation data, at least a bounding box that is a drawing area of the character of the sentence data, and the origin calculation process Character drawing position correction that sets drawing data including the character origin and the next origin which is the origin of the next character, the side bearing of the character, and the character width of the character as the character attribute data for the document data. Perform the setting process,
(3) For adjacent characters in the document data, it is determined whether or not the origin and the next origin match. If it is determined that they match, the adjacent characters are characters in the same sentence. An information processing apparatus that performs the same sentence setting process for setting the document data as the character attribute data indicating the above . The font collation means
(1) The size of the bounding box of the font in the font collation data corresponding to the character recognized by the character recognition means is calculated, and the font type and character size of the recognized character are estimated. And
(2) When the plurality of fonts or the plurality of character sizes in the font collation data are estimated, the recognized character is compared with one or both of the plurality of fonts and the plurality of character sizes. To calculate the similarity,
(3) The font and the character size having the highest similarity are selected, and the font and the character size when the similarity of the selected font and the character size with respect to the recognized character is equal to or greater than a predetermined threshold value. 2. The method according to claim 1, wherein it is determined that the character size is matched and collated, and when the similarity is lower than the predetermined threshold, it is determined that the corresponding font and the character size are absent and the character is not collated. The information processing apparatus described. When it is determined by the font collating means that the corresponding font and the character size are not present and collation is not performed, the character information in the document data is deleted for the area of the image data recognized as the character. And character image switching means for replacing with line drawing data.
  The information processing apparatus according to claim 2. In an information processing system comprising image data input in an image forming apparatus and a server that changes the image data to document data,
The server
Storage means for storing the image data and font collation data of a plurality of fonts and fonts of character sizes;
Character recognition means for recognizing characters from the image data stored in the storage means ;
The area of the image data is recognized as the character by the character recognition means, recognized the character, fall into one of a plurality of fonts and a plurality of character sizes of the font within the collation data stored in said storage means Font matching means for checking or matching,
The font set for the document data to the font and the character size is verified by collating means as character attribute data, and said drawing position location of the character as the character attribute data so as to correspond to the image data Character attribute data setting means for setting the document data ,
The character attribute data setting means includes:
(1) The font and the character size in the collated font collation data are compared with the image of the character of the document data to calculate the best matching coordinate, and the character is based on the coordinate. Perform the origin calculation process to calculate the origin of
(2) Designating the font and the character size in the collated font collation data, at least a bounding box that is a drawing area of the character of the sentence data, and the origin calculation process Character drawing position correction that sets drawing data including the character origin and the next origin which is the origin of the next character, the side bearing of the character, and the character width of the character as the character attribute data for the document data. Perform the setting process,
(3) For adjacent characters in the document data, it is determined whether or not the origin and the next origin match. If it is determined that they match, the adjacent characters are characters in the same sentence. An information processing system for performing the same sentence setting process for setting the document data as the character attribute data indicating the above .

Images