JP7230331B2 - Management system, information processing system, management method and program - Google Patents

Description

The present invention relates to a management system, an information processing system, a management method and a program.

2. Description of the Related Art In recent years, an increasing number of electronic devices are equipped with touch screen displays in order to facilitate input operations by users.

In addition to giving operation instructions to electronic devices, input operations using touch screen displays are also used for inputting documents by handwriting function). Therefore, an electronic device having a handwriting input function and a character recognition function can obtain text data corresponding to a handwritten input document (see, for example, Patent Document 1).

In a conventional electronic device, a user uses a handwriting input function to input handwriting in a predetermined entry field. If the determined entry field is narrow, handwriting (strokes) input by the user using the handwriting input function may extend beyond the entry field. When such strokes input by handwriting are converted into text, there is a problem that conventional electronic devices may not be able to input text data having a character size suitable for the size of the entry field.

An object of the embodiments of the present invention is to provide a management system capable of converting input stroke information into text with a character size corresponding to an entry column of a document.

In order to achieve the above-described problems, claim 1 of the present application is a management system capable of communicating with a terminal via a network, comprising document identification information for identifying a document, and one or more documents requiring handwriting input in the document. first storage means for storing area information of entry fields in association with each other; reception means for receiving from said terminal said document identification information and stroke information inputted on said terminal; said reception means textualizing the stroke information based on the area information of the one or more entry fields stored in the first storage means corresponding to the document identification information received by and the stroke information received by the receiving means and text converting means for converting the stroke information into text with the character size determined by the determining means, wherein the determining means receives the The start position and end position of the first stroke of the stroke information from the one or more entry fields based on the start position and end position of the first stroke of the stroke information and the area information of the one or more entry fields The entry field in which the position is entered is selected as the entry field in which the stroke information is input , and the size of the entry field in the character input direction is divided by the number of characters when the stroke information is converted into text. The character size is compared with a second character size in the character input direction and the vertical direction of the entry field, and the smaller one is determined as the character size after text conversion.

According to the embodiment of the present invention, input stroke information can be converted into text with a character size corresponding to the entry column of the document.

1 is a configuration diagram of an example of an information processing system according to a first embodiment; FIG. 3 is a hardware configuration diagram of an example of a transmission terminal according to the present embodiment; FIG. 1 is a hardware configuration diagram of an example of a computer according to the embodiment; FIG. 1 is a functional configuration diagram of an example of an information processing system according to a first embodiment; FIG. FIG. 4 is an image diagram of an example of a document displayed by a transmission terminal; 4 is a configuration diagram of an example of information about documents stored in a material information DB; FIG. 1 is a conceptual diagram of an example of a document layer, a text layer and a stroke layer; FIG. FIG. 10 is a sequence diagram of an example of processing when a user writes a stroke; It is a figure for demonstrating the process which recognizes the size of an entry column. FIG. 10 is a diagram for explaining a process of specifying an entry field in which strokes are described; 4 is a configuration diagram of an example of stroke information; FIG. FIG. 4 is a sequence diagram of an example including a procedure for a customer to handwrite a document; 4 is a configuration diagram of an example of data stored in a customer information DB; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
<System configuration>
FIG. 1 is a configuration diagram of an example of an information processing system according to the first embodiment. In the information processing system 1 of FIG. 1, a management system 10 and one or more transmission terminals 20 are communicably connected via a communication network 30 such as the Internet or a LAN. Also, the management system 10 is configured to have a transmission management system 12 and an information management system 14 . The transmission management system 12 is configured to have one or more transmission management server devices 16 . The information management system 14 is configured to have one or more information management server devices 18 .

The information management server device 18 stores and manages information of customers who are users who operate the transmission terminals 20, documents to be displayed on the transmission terminals 20, information related to the documents, and the like. A document stored and managed by the information management server device 18 is provided with one or more entry fields that require handwritten input.

The transmission terminal 20 receives the document stored in the information management server device 18 via the transmission management system 12 and displays it. The transmission terminal 20 has a handwriting input function, and accepts input of stroke information in the entry fields of the displayed document from the user. The transmission terminal 20 transmits to the transmission management server device 16 the stroke information that has been input by the user. Note that the stroke information is coordinate information of a point group indicating the trajectory of each stroke input by handwriting by the user when handwriting input is performed on the transmission terminal 20 . In addition, the transmission terminal 20 displays the text converted from the stroke information in the transmission management server device 16 in the entry column of the document as will be described later. The transmission terminal 20 is an example of a terminal that transmits stroke information input by the handwriting input function.

The transmission management server device 16 transmits the document stored in the information management server device 18 to the transmission terminal 20 . The transmission management server device 16 receives the stroke information input by the transmission terminal 20 from the user. The transmission management server device 16 determines the character size of the received stroke information as will be described later, converts the stroke information into text with that character size, and displays it in the entry column of the document.

Note that the transmission terminal 20 can be implemented by, for example, a videoconference terminal. The transmission management server device 16 can be implemented by, for example, a videoconference server device. The transmission management server device 16 and the transmission terminal 20 transmit and receive video and audio data related to the videoconference.

The transmission management server device 16 constituting the transmission management system 12 may be composed of a plurality of server devices, and any server device may be provided with the function. Further, the information management server device 18 that constitutes the information management system 14 may be composed of a plurality of server devices, and any server device may be provided with the function.

The configuration of the information processing system 1 shown in FIG. 1 is an example, and it goes without saying that there are various system configuration examples depending on the application and purpose. For example, in the management system 10 of the information processing system 1 in FIG. 1, the configuration in which the transmission management system 12 and the information management system 14 are separated is shown as an example. may be configured as Further, the information processing system 1 of FIG. 1 has a configuration in which at least part of the functions of the transmission management system 12 are provided in the information management system 14 or another information processing system, or at least part of the functions of the information management system 14 are provided in the transmission management system. 12 or other information processing system.

<Hardware configuration>
《Transmission terminal》
FIG. 2 is a hardware configuration diagram of an example of a transmission terminal according to this embodiment. As shown in FIG. 2, the transmission terminal 20 of this embodiment is configured to have a call terminal 100 and a display 120 .

The communication terminal 100 includes a CPU (Central Processing Unit) 101 that controls the overall operation of the communication terminal 100, a ROM (Read Only Memory) 102 that stores programs such as an IPL (Initial Program Loader) used to drive the CPU 101, and a CPU 101. A RAM (Random Access Memory) 103 used as a work area, a flash memory 104 for storing various data such as programs for the call terminal 100, image data, and audio data, and various data to the flash memory 104 under the control of the CPU 101. An SSD (Solid State Drive) 105 that controls reading or writing, a media drive 107 that controls reading or writing (storage) of data to a recording medium 106 such as a flash memory, and a destination of the call terminal 100 are selected. a power switch 109 for switching ON/OFF the power of the call terminal 100;

The call terminal 100 also includes a built-in camera 112 that captures an image of a subject under the control of the CPU 101 to obtain image data, an image sensor I/F 113 that controls driving of the camera 112, a built-in microphone 114 that inputs sound, A built-in speaker 115 that outputs audio, an audio input/output I/F 116 that processes input/output of audio signals between the microphone 114 and the speaker 115 under the control of the CPU 101, and an external display 120 under the control of the CPU 101 a display I/F 117 for transmitting various data, an external device connection I/F 118 for connecting various external devices, and an address for electrically connecting the above components as shown in FIG. A bus line 110 such as a bus or a data bus is provided.

Display 120 is an example of a display device having a handwriting input function. Display 120 is connected to display I/F 117 by cable 120c. The handwriting input function of the display 120 includes a contact sensor that detects contact with the display 120 by an electronic pen, a user's hand, or the like. The contact sensor can output the trajectory of the contact position of the electronic pen, user's hand, or the like as a handwriting input (stroke). In this way, the transmission terminal 20 can acquire stroke information input by the handwriting input function of the display 120 .

The camera 112 includes a lens and a solid-state imaging device that converts light into electric charge and digitizes an image (video) of a subject. etc. are used.

External devices such as an external camera, an external microphone, and an external speaker can be electrically connected to the external device connection I/F 118 via a USB (Universal Serial Bus) cable or the like. When an external camera is connected, the external camera is driven according to the control of the CPU 101 prior to the built-in camera 112 . Similarly, when an external microphone is connected or when an external speaker is connected, according to the control of the CPU 101, the external microphone and speaker are prioritized over the built-in microphone 114 and built-in speaker 115 respectively. Powered by an external speaker.

The recording medium 106 is detachably attached to the call terminal 100 . The flash memory 104 is an example of a non-volatile memory that reads or writes data under the control of the CPU 101, and may be an EEPROM (Electrically Erasable and Programmable ROM) or the like.

"Computer"
The transmission management server device 16 and the information management server device 18 are implemented by, for example, a computer having the hardware configuration shown in FIG. FIG. 3 is a hardware configuration diagram of an example of a computer according to this embodiment.

The computer of FIG. 3 includes a CPU 201 that controls the overall operation, a ROM 202 that stores programs used to drive the CPU 201 such as IPL, a RAM 203 that is used as a work area for the CPU 201, an HD 204 that stores various data such as programs, and the CPU 201. A HDD (Hard Disk Drive) 205 that controls reading or writing of various data to the HD 204 according to the control of , a media drive 207 that controls reading or writing (storage) of data to a recording medium 206 such as a flash memory, and displays various information. a display 208 for data communication, a network I/F 209 for data communication using the communication network 30, a keyboard 211 provided with a plurality of keys for input, a mouse 212, a CD-ROM as an example of a removable recording medium ( CD-ROM drive 214 for controlling reading or writing of various data to/from Compact Disc Read Only Memory) 213, and bus lines 210 such as an address bus and a data bus for electrically connecting the above components. there is

<Software configuration>
The information processing system 1 according to the first embodiment is implemented by, for example, the functional configuration shown in FIG. FIG. 4 is a functional configuration diagram of an example of the information processing system according to the first embodiment. In the functional configuration of FIG. 4, configurations unnecessary for the explanation of this embodiment are omitted as appropriate.

The information management system 14 of the information processing system 1 shown in FIG. 4 is realized by executing a program on one or more computers. The information management system 14 includes a transmission/reception section 71 , a storage/read processing section 72 , a customer information management section 73 , a document management section 74 and a storage section 75 . The storage unit 75 also has a customer information DB 76 , a material storage 77 and a material information DB 78 .

The customer information DB 76 stores data such as customer names and addresses. The material storage 77 stores documents such as application forms to which unique document IDs (an example of document identification information) are assigned. The material information DB 78 stores information on each document in association with the document ID. The transmission/reception unit 71 controls communication with the transmission management system 12 . The transmission/reception unit 71 transfers the received request to the customer information management unit 73 or the material management unit 74 according to the content.

The customer information management unit 73 reads data from the customer information DB 76 or writes data to the customer information DB 76 according to the content of the request. The material management unit 74 reads a document from the material storage 77 or writes a document to the material storage 77 according to the content of the request. In addition, the material management unit 74 reads information about the document from the material information DB 78 or writes information about the document to the material information DB 78 according to the content of the request. The customer information management unit 73 and the material management unit 74 perform read operations from the storage unit 75 and write operations to the storage unit 75 via the storage/read processing unit 72 .

Also, the transmission management system 12 of the information processing system 1 is implemented by executing a program on one or more computers. The transmission management system 12 has a storage unit 51 , a storage/read processing unit 52 , a conference control unit 53 , a transmission/reception unit 54 , a handwriting processing unit 55 and an OCR processing unit 56 . The storage unit 51 also has a stroke storage 57 . The transmission management system 12 is a system that manages transmission and reception of video and audio data relating to the video conference, display of stroke information, and the like.

Video and audio data for the videoconference are processed by the conference control section 53 via the transmission/reception section 54 . The stroke information from the transmission terminal 20 is transferred to the handwriting processing section 55 via the transmission/reception section 54 . Note that the stroke information represents the handwriting input by handwriting by the user using the handwriting input function. The handwriting processing unit 55 controls display of strokes as described later.

The OCR processing unit 56 performs OCR processing for converting stroke information into text such as numbers and characters. The stroke storage 57 of the storage unit 51 temporarily stores stroke information for OCR processing. Note that the conference control unit 53 , the handwriting processing unit 55 , and the OCR processing unit 56 perform reading operations from the storage unit 51 and writing operations to the storage unit 51 via the storage/reading processing unit 52 .

Furthermore, the transmission terminal 20 of the information processing system 1 is implemented by executing a program. The transmission terminal 20 has an operation input reception unit 61 , a display control unit 62 , a storage/read processing unit 63 , a storage unit 64 and a transmission/reception unit 65 . The transmission terminal 20 transmits video and audio data related to the videoconference to the transmission management system 12 via the transmission/reception unit 65 .

The operation input reception unit 61 recognizes stroke information, which is handwritten information input by the user using the handwriting input function, and transmits the stroke information to the transmission management system 12 via the transmission/reception unit 65 . The display control unit 62 performs display control of video data related to the video conference received via the transmission/reception unit 65 and reproduction control of audio data. A display control unit 62 controls the display of documents, and the display of strokes and texts on documents. Note that the operation input reception unit 61 and the display control unit 62 perform read operations from the storage unit 64 and write operations to the storage unit 64 via the storage/read processing unit 63 .

<Processing>
For example, the information processing system 1 according to the first embodiment accepts handwritten input from a user into a document with entry fields that require handwriting input, converts the strokes of the handwritten input into text, and converts the entry fields into text. to display.

《Document example》
FIG. 5 is an image diagram of an example of a document displayed by a transmission terminal. The document in FIG. 5 represents an image of an application form as an example. In the document shown in FIG. 5, entry fields such as name, address, and furigana are entry fields that require handwriting input. A document such as that shown in FIG. 5 is assigned a unique document ID and stored in the material storage 77 of the information management system 14 .

Next, the contents of the document-related information stored in the material information DB 78 of the information management system 14 will be described using the name entry column (name column) in the document of FIG. For example, as shown in FIG. 5, the right direction toward the document is set as the x-axis, and the upward direction as the y-axis.

Also, the coordinates of the upper left position of the entry column are defined as (x1, y1). The coordinates of the lower right position of the entry column are defined as (x2, y2). Here, the document ID of the document in FIG. 5 is defined as "001". Also, the entry field ID that uniquely identifies the name field of the document in FIG. 5 is defined as "100".

Further, the coordinates of the upper left position of the name column are defined as (100, 600). The coordinates of the bottom right position of the name column are defined as (200, 550). The entry field ID of the phonetic field above the name field is defined as "120", the coordinates of the upper left position of the phonetic field are defined as (100, 620), and the coordinates of the lower right position of the phonetic field are defined as (200, 550). defined as

Also, the character type (character type) to be input in the name field is defined as "Japanese". The character type (character type) input in the furigana field above the name field is defined as "hiragana". The information as described above is defined for all entry fields that require handwritten input in the document.

For example, in the case of the name column of the document in FIG. 5 and the furigana column above the name column, the information related to the document stored in the material information DB 78 is as shown in FIG. 6, for example. FIG. 6 is a configuration diagram of an example of information about documents stored in the material information DB. In the document information stored in the material information DB 78, for example, as shown in FIG. 6, coordinates of the upper left position and lower right position of the entry column and the character type are defined in association with the document ID and the entry column ID. ing.

As described above, the information about the document in FIG. 6 defines the areas of the entry fields of the document that require handwriting input and the character types to be entered in the entry fields. The information about the document is an example of the area information of the entry field.

《Example of document display》
In the transmission terminal 20, the document makes it possible to display strokes and text on the document using, for example, a document layer, a text layer, and a stroke layer as shown in FIG.

FIG. 7 is a conceptual diagram of an example of the document layer, text layer, and stroke layer. In the example of FIG. 7, a text layer is provided above the document layer, and a stroke layer is provided above the text layer. The document layer displays documents. The stroke layer displays strokes handwritten by the user. The text layer converts handwritten strokes into text and displays them.

The display of strokes and OCR processing for converting strokes into text are executed by the transmission management system 12 . The transmission management system 12 superimposes the image data of the document layer, the text layer, and the stroke layer associated with the document ID, combines them into one piece of screen data, and transmits it to the transmission terminal 20 .

The transmission terminal 20 displays a piece of screen data in which the image data of the document layer, the text layer, and the stroke layer associated with the document ID in the transmission management system 12 are combined, so that strokes and text on the document are displayed. can be displayed.

<<Processing when writing strokes by the user>>
FIG. 8 is a sequence diagram of an example of processing when a user writes a stroke. It is assumed that the user has operated the transmission terminal 20 and displayed the document before the process of the sequence diagram of FIG. Here, processing when the application form document of FIG. 5 is displayed on the transmission terminal 20 will be described.

When the user writes strokes on a document using the handwriting input function of transmission terminal 20, operation input reception unit 61 of transmission terminal 20 recognizes the coordinate information of the strokes. Recognition of the coordinate information is performed periodically, for example, in about 10 ms to 100 ms. The shorter the coordinate information recognition cycle, the smoother the stroke, but the larger the amount of data tends to be.

For example, when the user starts writing strokes and once the pen or finger is removed from the display 120 displaying the document, the operation input reception unit 61 transmits stroke information to the transmission management system 12 in step S11.

Based on the stroke information received from the transmission terminal 20, the handwriting processing unit 55 of the transmission management system 12 displays strokes in the stroke layer shown in FIG. Here, for example, one character such as Chinese characters may be composed of a plurality of strokes, and a trigger for starting the OCR process is required.

Therefore, in the present embodiment, the user starts to describe strokes, and once the pen or finger is removed from the display 120 that displays the document, the OCR process is started when no strokes are described for a certain period of time. do. This fixed time is assumed to be, for example, about 2 seconds to 5 seconds. In order to recognize the predetermined time, the handwriting processing unit 55 of the transmission management system 12 that has received the stroke information from the transmission terminal 20 resets the timer in step S12.

Proceeding to step S13 following step S12, the handwriting processing unit 55 displays strokes in the stroke layer based on the stroke information received from the transmission terminal 20. FIG. Proceeding to step S<b>14 , the conference control unit 53 periodically superimposes the image data of the document layer, the text layer, and the stroke layer, combines them into one piece of screen data, and transmits it to the transmission terminal 20 .

As for the transmission cycle of the screen data to the transmission terminal 20, for example, a 1/30 second cycle (30 fps) can be considered. Since the processing of steps S15 to S18 is the same as the processing of steps S11 to S14, description thereof is omitted.

After a certain period of time has elapsed since the stroke information was received from the transmission terminal 20, the transmission management system 12 recognizes in step S19 that a time-out has occurred due to, for example, interrupt processing by a timer, and starts stroke-to-text conversion processing. Start.

The conversion process from strokes to text will be described below. Proceeding to step S20, the handwriting processing unit 55 designates the document ID and requests the information management system 14 for information on the document. Proceeding to step S21, the handwriting processing unit 55 receives information on the document corresponding to the document ID specified in step S20.

Proceeding to step S22, the handwriting processing unit 55 acquires stroke information from the storage unit 51 via the storage/reading processing unit 52, and recognizes the position where the stroke is written (stroke position) based on the stroke information.

Proceeding to step S23, the handwriting processing unit 55 specifies an entry column (input column) in which strokes are described based on the information about the document received from the information management system 14 in step S21 and the stroke positions recognized in step S22. . Details of the processing in step S23 will be described later.

Proceeding to step S24, the handwriting processing unit 55 confirms the character type of the entry field identified in step S23 based on the "character type" of the information about the document received from the information management system 14 in step S21, and identifies the OCR type. do. For example, in the information related to the document in FIG. 6 corresponding to the document in FIG. 5, if the stroke description is in the name column, "Japanese" is specified as the OCR type.

Proceeding to step S25, the handwriting processing unit 55, based on the "coordinates of the upper left position and lower right position of the entry field" regarding the document received from the information management system 14 in step S21, Recognize size (area). Proceeding to step S26, the handwriting processing unit 55 determines the character size based on the size of the entry field recognized in step S25 and the number of characters after the OCR processing by the OCR processing unit 56. FIG. Note that the handwriting processing unit 55 can obtain the “number of characters after OCR processing” used for determining the character size in step S26 by causing the OCR processing unit 56 to perform provisional OCR processing.

Proceeding to step S27, the handwriting processing unit 55 requests the OCR processing unit 56 to perform OCR processing using the OCR type specified in step S24, and converts the stroke information into text of the character size determined in step S26. The handwriting processing unit 55 arranges the text converted to text by the OCR processing unit 56 so as to be displayed in the entry column corresponding to the position on the document layer where the stroke information is input, and displays it on the text layer shown in FIG. do. In this way, in step S27, the character type of the entry field specified in step S23 is confirmed, and OCR processing is performed on the premise of the character type, thereby improving the recognition accuracy.

In addition, proceeding to step S28, the handwriting processing unit 55 deletes the strokes displayed in the stroke layer of FIG. Through the processing up to this point, the conference control unit 53 generates a single screen data in which the image data of the document layer, the text layer, and the stroke layer associated with the document ID are combined, and the stroke input by handwriting by the user is displayed. It is possible to generate a piece of screen data in which the text that has been erased and converted into text from the stroke is displayed in the entry column of the document.

Proceeding to step S29, the conference control unit 53 transmits the generated single screen data to the transmission terminal 20, so that the stroke input by handwriting by the user is converted into text with a character size based on the entry column, and the transmission terminal 20 can be displayed.

In the sequence diagram of FIG. 8, the stroke information is transmitted from the transmission terminal 20 to the transmission management system 12 when the user starts writing strokes and once the pen or finger is removed from the display 120 that displays the document. I showed an example to do. If the transmission terminal 20 has a function of superimposing the image data of the document layer, the text layer, and the stroke layer as shown in FIG. The transmission of stroke information to the transmission management system 12 may be performed collectively.

<<Details of processing in steps S25 to S28>>
Next, details of the processing of steps S25 to S28 in FIG. 8 will be described. FIG. 9 is a diagram for explaining the process of recognizing the size of the entry field. Based on the "coordinates of the upper left position and lower right position of the entry field" of the information about the document received from the information management system 14 in step S21, the handwriting processing unit 55 changes the size (region) of the entry field identified in step S23 to: It is calculated by the formulas (1) and (2). Let Lx be the size of the entry field in the horizontal direction (x direction) in equation (1), and let Ly be the size of the entry field in the vertical direction (y direction) in equation (2).

Lx=x2-x1 (1)
Ly=y1−y2 (2)
In the example of FIG. 9, the ``number of characters after OCR processing'' is 4 characters of ``Kiguchi Taro''. The handwriting processing unit 55 determines the character size as follows using the following equations (3) and (4). The formula (3) divides the size of the entry field in the horizontal direction (character input direction of the entry field) by the number of characters after OCR processing (the number of characters when stroke information is converted to text).

"Character size 1" = Lx / Number of characters (3)
"Character size 2" = Ly (4)
Then, the handwriting processing unit 55 compares "character size 1" and "character size 2" and determines the smaller one as the character size after text conversion. Then, the handwriting processing unit 55 arranges the converted text at a position corresponding to the entry column in the text layer. Note that the strokes displayed in the stroke layer become unnecessary after the text converted to text is arranged in the position corresponding to the entry field in the text layer. Therefore, the handwriting processor 55 deletes the strokes displayed in the stroke layer, and also deletes the stroke information in the stroke storage 57 .

<<Details of processing in step S23>>
Next, details of the processing of step S23 in FIG. 8 will be described. 10A and 10B are diagrams for explaining the process of specifying an entry column in which strokes are described. FIG. 10A is an image diagram of an example of strokes written on a document. In this embodiment, as shown in FIG. 10B, the handwriting processing unit 55 sets the start position (xs1, ys1) and the end position (xs2, ys2) of the first stroke of the stroke to the coordinates (x1, ys2) of the upper left position. y1) and coordinates (x2, y2) of the lower right position is determined by the following equation (5).

x1<xs1, xs2<x2 and y1>ys1, ys2>y2 (5)
When the expression (5) is satisfied, the handwriting processing unit 55 can identify the entry column in which the stroke is described by determining that the entry is handwritten input to the entry column. The specification of such entry fields in which strokes are described is an example of processing for selecting entry fields corresponding to positions on the document layer where stroke information is input.

By specifying the entry field in this way, in the present embodiment, even if the entry field is small, if the stroke is written so that the first stroke fits in the entry field, even if the stroke protrudes out of the entry field, , can be converted to text of a character size that fits in the size of the entry field. On the other hand, even if the entry field is large, if the stroke is described so that the first stroke fits in the entry field, even if the stroke is small relative to the size of the entry field, the size of the entry field is suitable. Can be converted to font size text. As described above, according to this embodiment, the user can comfortably perform handwriting input in the entry fields of the document.

Note that the stroke information has a configuration as shown in FIG. 11, for example. FIG. 11 is a configuration diagram of an example of stroke information. The stroke information in FIG. 11 includes SEQ, operation name, IP address and Port No. of the transmission terminal 20 which is the transmission source of the stroke information. , an operation type, an operation target describing stroke information, and data indicating the contents of the stroke information as items.

For example, SEQ "1" indicates the contents of strokes drawn on the document with the document ID "001" on the transmission terminal 20 (stroke data representing handwriting input by the user by a plurality of coordinates).

<<Usage example of handwriting input according to the present embodiment>>
For example, there is a remote customer service system in which a customer visiting a store and an operator at an operation center outside the store have a conversation (customer service) using a video conference system. In such a remote customer service system, a tablet terminal or the like capable of handwriting input is used, and it is assumed that the operator and the customer share materials, and the customer handwrites an application form or contract.

In the following, an example in which a customer visiting a store for the first time inputs handwritten data into a customer management document and an application form, which are examples of documents, will be described. FIG. 12 is an example sequence diagram including a procedure for a customer to handwrite a document.

In steps S51 to S53, the handwritten strokes input by the customer into the document for customer management are converted into text in a character size based on the entry column and displayed on the transmission terminal 20 by the same processing as in steps S11 to S29 of FIG. Proceeding to step S54, the customer completes the entry in the customer management document.

Proceeding to step S55, the transmission management system 12 instructs the information management system 14 to perform customer information search processing based on the text information such as the name and address converted into text by the OCR conversion processing. The information management system 14 searches the customer information DB 76 based on the text information such as name and address.

Here, since the customer is the first visitor, the customer information is not registered in the customer information DB 76 . Proceeding to step S56, the transmission management system 12 instructs the information management system 14 to perform customer information registration processing based on the text information such as the name and address converted into text by the OCR conversion processing. The information management system 14 issues a new customer ID and registers it in the customer information DB 76 in association with text information such as name and address converted into text by OCR conversion processing.

After that, in steps S57 to S59, the strokes handwritten by the customer on the application form are converted into text in a character size based on the entry column and displayed on the transmission terminal 20 by the same processing as in steps S11 to S29 of FIG. Proceeding to step S60, the customer completes the application form.

Proceeding to step S61, the transmission management system 12 superimposes the image data of the document layer and the text layer shown in FIG. 7, and performs a documenting process of converting them into one document file. A document file is a PDF file or an image file such as a JPEG file or a PNG file. When the documenting process is completed, the transmission management system 12 instructs the information management system 14 to register the document. The information management system 14 associates and registers the document file for which document registration has been instructed and the customer ID.

FIG. 13 is a configuration diagram of an example of data stored in the customer information DB. In the customer information search process at step S55, the customer information DB shown in FIG. 13 is searched using the converted text of the customer information. For example, in the customer information DB of FIG. 13, the customer's name and telephone number are used as search keys.

Here, it is assumed that the name entered by the customer is "Taro Kikuchi" and the telephone number is "03-000-0000". In this case, since the customer is not registered in the customer information DB 76, a new customer ID is issued and the name, address, telephone number and E-mail are registered. Thereafter, in a teleconference between a customer and an operator, by using this customer ID, the operator can recognize the target customer for whom the teleconference is being held.

Consider a case where a customer fills out some kind of contract. The customer writes in the contract by hand using the same procedure as in the sequence diagram of FIG. The transmission management system 12 performs OCR conversion processing on the stroke information. When all the entries in the contract are completed, the transmission management system 12 superimposes the image data of the document layer and the text layer shown in FIG. instruct.

The information management system 14 associates the document file for which document registration is instructed with the customer ID and registers them in the material storage 77 . As a result, when this customer visits the store next time, it is possible to check the history of the contract. Furthermore, even if the customer visits a store different from the one used this time, information such as contract history can be shared.

According to this embodiment, the content handwritten by the customer in the entry field of the document is converted into text and the customer information DB 76 can be automatically updated. You can reduce the burden.
[Other embodiments]
Note that the information processing system 1 according to the first embodiment described above is an example, and can be widely applied to information processing systems such as a video conference system that allows handwriting input to shared materials. Video conferencing systems that allow handwriting input to shared materials include video conferencing systems that use electronic blackboards and tablet terminals.

The login from the transmission terminal 20 to the transmission management system 12 may be by the terminal D or by the user ID. Further, the transmission terminal 20 may be a terminal dedicated to the video conference system, or may be a PC, a tablet terminal, or the like that operates Web contents on a browser that performs the same processing as the transmission terminal 20 of the first embodiment.

Each function of the embodiments described above may be implemented by one or more processing circuits. Here, the "processing circuit" in this specification means a processor programmed by software to perform each function, such as a processor implemented by an electronic circuit, or a processor designed to perform each function described above. ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array), SOC (System on a chip), GPU (Graphics Processing Unit), and devices such as conventional circuit modules.

The invention is not limited to the specifically disclosed embodiments above, but various modifications and changes are possible without departing from the scope of the claims. Note that the information processing system 1 described in the present embodiment is only an example, and it goes without saying that there are various system configuration examples according to usage and purpose.

The material information DB 78 is an example of the first storage means described in the claims. The transmitting/receiving unit 54 is an example of receiving means and transmitting means. The handwriting processing unit 55 is an example of determining means and arranging means. The OCR processing section 56 is an example of a text conversion means. The material storage 77 is an example of the second storage means. The coordinates of the upper left position and the lower right position of the entry field of the information about the document are an example of the area information of the entry field. The management method executed by the management system 10 that can communicate with the transmission terminal 20 via the network 30 is executed by the management system that can communicate with the terminal that transmits the stroke information input by the handwriting input function via the network. It is an example of a management method.

The process of storing information on each document in association with the document ID in the material information DB 78 is an example of the storage step. The process in which the transmission management system 12 receives the stroke information from the transmission terminal 20 in step S11 or the like is an example of the receiving step. The process of step S26 of determining the character size based on the recognized size of the entry field and the number of characters after OCR processing by the OCR processing unit 56 is an example of the determination step.

The process of requesting the OCR processing unit 56 to perform OCR processing in the OCR type specified in step S24 and converting the stroke information into text of the character size determined in step S26 is an example of the text conversion step. The process of arranging the converted text so as to be displayed in the entry column corresponding to the position on the document layer where the stroke information is input and displaying it on the text layer shown in FIG. 7 is an example of the arrangement step.

1 information processing system 10 management system 12 transmission management system 14 information management system 16 transmission management server device 18 information management server device 20 transmission terminal 30 communication network 51 storage unit 52 storage/read processing unit 53 conference control unit 54 transmission/reception unit 55 handwriting processing Unit 56 OCR processing unit 57 Stroke storage 61 Operation input reception unit 62 Display control unit 63 Storage/read processing unit 64 Storage unit 65 Transmission/reception unit 71 Transmission/reception unit 72 Storage/read processing unit 73 Customer information management unit 74 Material management unit 75 Storage unit 76 Customer information database
77 Material storage 78 Material information DB

JP 2015-90670 A

Claims (8)

A management system capable of communicating with a terminal via a network,
a first storage means for storing document identification information for identifying a document in association with area information for one or more entry fields for which handwriting input is required in the document;
receiving means for receiving the document identification information and the stroke information entered on the terminal from the terminal;
The stroke information is based on the area information of the one or more entry fields stored in the first storage means corresponding to the document identification information received by the receiving means, and the stroke information received by the receiving means. A determination means for determining the character size when the is converted into text;
text conversion means for converting the stroke information into text with the character size determined by the determination means;
has
Based on the start position and the end position of the first stroke of the stroke information received by the receiving means and the area information of the one or more entry fields, the determination means selects from the one or more entry fields the The entry field containing the start position and end position of the first stroke of the stroke information is selected as the entry field in which the stroke information is input , and the size of the entry field in the character input direction is set according to the stroke information. A first character size divided by the number of characters in text conversion is compared with a second character size in the direction perpendicular to the character input direction in the entry field, and the smaller one is determined as the character size after text conversion. A management system characterized by : arrangement means for arranging the text converted from the stroke information by the text conversion means so as to be displayed in the selected entry field;
2. The management system of claim 1, further comprising: transmission means for transmitting screen data to the terminal;
further having
The arrangement means generates the screen data in which the text and the document are superimposed so that the text converted from the stroke information by the text conversion means is displayed in the selected entry column. 3. Management system according to claim 2. second storage means for storing the document in which the text converted from the stroke information by the text conversion means is superimposed so as to be displayed in the selected entry field;
4. The management system according to any one of claims 1 to 3 , further comprising: A management method executed by a management system capable of communicating with a terminal via a network,
a storage step of associating document identification information for identifying a document with area information of one or more entry fields in the document that require handwriting input and storing the information in a first storage means;
a receiving step of receiving from the terminal the document identification information and stroke information entered on the terminal;
The stroke information is based on the area information of the one or more entry fields stored in the first storage means corresponding to the document identification information received in the receiving step, and the stroke information received in the receiving step. A determination step for determining the character size when converting to text;
a textualization step of textualizing the stroke information with the character size determined by the determination step;
has
The determining step selects from the one or more entry fields based on the start position and the end position of the first stroke of the stroke information received in the receiving step and the area information of the one or more entry fields. The entry field containing the start position and end position of the first stroke of the stroke information is selected as the entry field in which the stroke information is input , and the size of the entry field in the character input direction is set according to the stroke information. A first character size divided by the number of characters in text conversion is compared with a second character size in the direction perpendicular to the character input direction in the entry field, and the smaller one is determined as the character size after text conversion. A management method characterized by A management system that can communicate with terminals via a network,
a first storage means for storing document identification information for identifying a document in association with area information for one or more entry fields for which handwriting input is required in the document;
receiving means for receiving the document identification information and the stroke information entered on the terminal from the terminal;
The stroke information is based on the area information of the one or more entry fields stored in the first storage means corresponding to the document identification information received by the receiving means, and the stroke information received by the receiving means. Determining means for determining the character size when converting to text,
textualization means for textualizing the stroke information in the character size determined by the determination means;
function as
Based on the start position and the end position of the first stroke of the stroke information received by the receiving means and the area information of the one or more entry fields, the determination means selects from the one or more entry fields the The entry field containing the start position and end position of the first stroke of the stroke information is selected as the entry field in which the stroke information is input , and the size of the entry field in the character input direction is set according to the stroke information. A first character size divided by the number of characters in text conversion is compared with a second character size in the direction perpendicular to the character input direction in the entry field, and the smaller one is determined as the character size after text conversion. A program characterized by : An information processing system having a terminal and a management device capable of communicating with the terminal via a network,
a first storage means for storing document identification information for identifying a document in association with area information for one or more entry fields for which handwriting input is required in the document;
receiving means for receiving the document identification information and the stroke information entered on the terminal from the terminal;
The stroke information is based on the area information of the one or more entry fields stored in the first storage means corresponding to the document identification information received by the receiving means, and the stroke information received by the receiving means. A determination means for determining the character size when the is converted into text;
text conversion means for converting the stroke information into text with the character size determined by the determination means;
has
Based on the start position and the end position of the first stroke of the stroke information received by the receiving means and the area information of the one or more entry fields, the determination means selects from the one or more entry fields the The entry field containing the start position and end position of the first stroke of the stroke information is selected as the entry field in which the stroke information is input , and the size of the entry field in the character input direction is set according to the stroke information. A first character size divided by the number of characters in text conversion is compared with a second character size in the direction perpendicular to the character input direction in the entry field, and the smaller one is determined as the character size after text conversion. An information processing system characterized by : A management method executed in an information processing system having a terminal and a management device capable of communicating with the terminal via a network,
a storage step of associating document identification information for identifying a document with area information of one or more entry fields in the document that require handwriting input and storing the information in a first storage means;
a receiving step of receiving from the terminal the document identification information and stroke information entered on the terminal;
The stroke information is based on the area information of the one or more entry fields stored in the first storage means corresponding to the document identification information received in the receiving step, and the stroke information received in the receiving step. A determination step for determining the character size when converting to text;
a textualization step of textualizing the stroke information with the character size determined by the determination step;
has
The determining step selects from the one or more entry fields based on the start position and the end position of the first stroke of the stroke information received in the receiving step and the area information of the one or more entry fields. The entry field containing the start position and end position of the first stroke of the stroke information is selected as the entry field in which the stroke information is input , and the size of the entry field in the character input direction is set according to the stroke information. A first character size divided by the number of characters in text conversion is compared with a second character size in the direction perpendicular to the character input direction in the entry field, and the smaller one is determined as the character size after text conversion. A management method characterized by

Images