JP6499198B2 - Electronic device and method for handwriting - Google Patents

Description

  The embodiments described herein relate to techniques for processing handwritten document information.

  In recent years, various electronic devices such as tablets, PDAs, and smartphones have been developed. Many electronic devices of this type are equipped with a touch screen display to facilitate an input operation by a user.

  Recently, electronic devices capable of handling handwritten character strings have also been developed.

  The user can easily take notes by using an electronic device that supports handwriting input.

JP 2007-265171 A

  However, in the past, no technology has been considered for enabling smooth handwriting input.

  The objective of one form of this invention is to provide the electronic device and method which can enable smooth handwriting input.

According to the embodiment, the electronic device includes display control means and processing means. The display control means displays a first stroke input by handwriting on the screen. The processing means displays a plurality of handwriting candidates that are input candidates determined from the first stroke on the screen according to the priority of the plurality of handwriting candidates when the first stroke is input. Execute the process. The handwriting candidate list box of n rows × m columns (n and m are integers of 2 or more), which is a display area of the plurality of handwriting candidates, is displayed in a first column displayed at a position corresponding to the input position of the first stroke. M regions including a first region and a second region arranged side by side from the first region. The processing means has a first position in the first region in which the first handwriting candidate with the highest priority among the plurality of handwriting candidates is in the vicinity of the input position of the first stroke in the handwriting candidate list box. And the second highest-priority second handwriting candidate among the plurality of handwriting candidates is arranged in the column direction with the first position of the handwriting candidate list box. displayed in, k-th high priority of the k handwriting candidate of the plurality of handwriting candidate (n <k ≦ 2n) are arranged next to next to the first region of the handwriting candidate list box, the first Displayed at the k-th k-th position in the two areas.

FIG. 1 is an exemplary perspective view showing an appearance of an electronic apparatus according to the embodiment. FIG. 2 is an exemplary diagram showing a cooperative operation between the electronic device of FIG. 1 and an external device. FIG. 3 is a diagram showing an example of a handwritten document handwritten on the touch screen display of the electronic apparatus of FIG. FIG. 4 is an exemplary diagram for explaining time-series information generated by the electronic apparatus of FIG. FIG. 5 is an exemplary block diagram showing a system configuration of the electronic apparatus of FIG. FIG. 6 is an exemplary block diagram illustrating a functional configuration of a handwritten note application program executed by the electronic device of FIG. FIG. 7 is an exemplary diagram showing a user interface (editing screen) displayed by the electronic apparatus of FIG. FIG. 8 is an exemplary diagram showing a handwriting candidate list box displayed in the edit view area in the user interface. FIG. 9 is an exemplary diagram showing an arrangement order of a plurality of handwriting candidates in the handwriting candidate list box of FIG. FIG. 10 is a diagram showing another example of the handwriting candidate list box displayed in the edit view area. FIG. 11 is an exemplary diagram for explaining a process of complementing a temporary stroke with a selected handwriting candidate in the handwriting candidate list box of FIG. FIG. 12 is an exemplary diagram for explaining a process of displaying a handwriting candidate list box below the provisional stroke input position. FIG. 13 is an exemplary diagram showing an arrangement order of a plurality of handwriting candidates in the handwriting candidate list box of FIG. FIG. 14 is an exemplary diagram showing the relationship between the structure of the handwriting candidate list box displayed in the right-handed mode and the structure of the handwriting candidate list box displayed in the left-handed mode. FIG. 15 is an exemplary diagram showing the relationship between the temporary stroke input position and the display position of the handwriting candidate list box in the left-handed mode. FIG. 16 is an exemplary diagram illustrating a state in which the same handwriting candidate as the handwriting candidate displayed last time is displayed at the same place in the handwriting candidate list box. FIG. 17 is an exemplary diagram illustrating an example of a relationship between a temporary stroke and a handwriting candidate list box. FIG. 18 is an exemplary diagram showing how handwriting candidates in the handwriting candidate list box are changed as input of temporary strokes continues. FIG. 19 is an exemplary diagram for explaining a process of automatically changing the starting point of the temporary stroke sequence used for predicting the handwriting candidate while the input of the temporary stroke is continued. FIG. 20 is an exemplary diagram showing a state where a handwriting candidate in the handwriting candidate list box is selected. FIG. 21 is an exemplary diagram for explaining a process of complementing a temporary stroke sequence with selected handwriting candidates. FIG. 22 is an exemplary flowchart showing a procedure of automatic complement processing executed by the electronic apparatus of FIG. FIG. 23 is an exemplary flowchart showing a procedure of handwriting candidate list box display processing executed by the electronic apparatus of FIG. FIG. 24 is a flowchart illustrating another example of the handwriting candidate list box display process.

Hereinafter, embodiments will be described with reference to the drawings.
FIG. 1 is a perspective view illustrating an external appearance of an electronic apparatus according to an embodiment. This electronic device is, for example, a pen-based portable electronic device that can be input by handwriting with a pen (stylus) or a finger. This electronic device can be realized as a tablet computer, a notebook personal computer, a smartphone, a PDA, or the like. Below, the case where this electronic device is implement | achieved as the tablet computer 10 is assumed. The tablet computer 10 is a portable electronic device that is also called a tablet or a slate computer. As shown in FIG. 1, the tablet computer 10 includes a main body 11 and a touch screen display 17. The main body 11 has a thin box-shaped housing. The touch screen display 17 is attached so as to overlap the upper surface of the main body 11.

  The touch screen display 17 incorporates a flat panel display and a sensor. The sensor is configured to detect a contact position between the screen of the flat panel display and the pen or the finger. The flat panel display may be, for example, a liquid crystal display (LCD). As the sensor, for example, a capacitive touch panel, an electromagnetic induction digitizer, or the like can be used. In the following, although not limited, it is assumed that the touch panel is incorporated in the touch screen display 17.

  A touch panel is arrange | positioned on the screen of a flat panel display, for example. The touch screen display 17 can detect not only a touch operation on a screen using a finger but also a touch operation on a screen using the pen 100.

  The user can perform a handwriting input operation on the touch screen display 17 using an external object (the pen 100 or a finger). During the handwriting input operation, the movement trajectory of the external object (the pen 100 or the finger) on the screen, that is, the handwriting is drawn in real time. The trajectory of the movement of the external object while the external object is in contact with the screen corresponds to one stroke. A set of many strokes corresponding to a handwritten character or a handwritten figure constitutes a handwritten document.

  In the present embodiment, this handwritten document is stored in the storage medium as time-series information indicating the coordinate sequence of the trajectory of each stroke and the order relationship between the strokes, not image data. Details of this time-series information will be described later with reference to FIG. 4, but this time-series information includes a plurality of stroke data respectively corresponding to a plurality of strokes. Each stroke data corresponds to a certain stroke, and includes a coordinate data series (time series coordinates) corresponding to each point on the stroke. The order of arrangement of the stroke data corresponds to the order in which the strokes are handwritten, that is, the stroke order.

  The tablet computer 10 can read existing arbitrary time-series information (handwritten document information) from the storage medium, and can display handwriting (a plurality of strokes) indicated by the handwritten document information on the screen. Furthermore, the tablet computer 10 has an editing function. This editing function can delete or move an arbitrary stroke or an arbitrary handwritten character in the displayed handwritten document by using the “eraser” tool, the “range selection” tool, and various other tools. Further, this editing function includes a function for canceling a history of some handwriting operations.

  Furthermore, the tablet computer 10 also has an automatic complement (stroke recommendation) function. This auto-completion function is a function for assisting a user's handwriting input operation so that a large number of character strings can be easily input by handwriting.

  FIG. 2 shows an example of cooperative operation between the tablet computer 10 and an external device. The tablet computer 10 can cooperate with the personal computer 1 and the cloud. That is, the tablet computer 10 includes a wireless communication device such as a wireless LAN, and can execute wireless communication with the personal computer 1. Furthermore, the tablet computer 10 can also execute communication with the server 2 on the Internet. The server 2 may be a server that executes an online storage service and other various cloud computing services.

  The personal computer 1 includes a storage device such as a hard disk drive (HDD). The tablet computer 10 can transmit time-series information (handwritten document) to the personal computer 1 via the network and record it on the HDD of the personal computer 1 (upload).

  Accordingly, even when the storage capacity of the tablet computer 10 is small, the tablet computer 10 can handle a large amount of time-series information (handwritten document) or a large amount of time-series information (handwritten document).

  Furthermore, the tablet computer 10 can read any one or more handwritten documents recorded in the HDD of the personal computer 1 (download). The tablet computer 10 can display each stroke indicated by the read handwritten document on the screen of the touch screen display 17 of the tablet computer 10.

  Further, the destination to which the tablet computer 10 communicates may not be the personal computer 1 but the server 2 on the cloud that provides a storage service as described above. The tablet computer 10 can transmit a handwritten document to the server 2 via the network and record it in the storage device 2A of the server 2 (upload). Furthermore, the tablet computer 10 can read an arbitrary handwritten document recorded in the storage device 2A of the server 2 (download). The tablet computer 10 can display each stroke indicated by the read handwritten document on the screen of the touch screen display 17 of the tablet computer 10.

  Thus, in the present embodiment, the storage medium in which the handwritten document is stored may be any one of the storage device in the tablet computer 10, the storage device in the personal computer 1, and the storage device in the server 2.

  Next, with reference to FIG. 3 and FIG. 4, the relationship between the handwritten stroke (character, mark, figure (line diagram), table, etc.) handwritten by the user and the handwritten document will be described. FIG. 3 shows an example of a handwritten character string handwritten on the touch screen display 17 using the pen 100 or the like.

  In a handwritten document, there are many cases where another character or graphic is handwritten on the character or graphic once handwritten. In FIG. 3, the handwritten character string “ABC” is handwritten in the order of “A”, “B”, and “C”, and then the handwritten arrow is handwritten in the immediate vicinity of the handwritten character “A”. The case is envisaged.

  The handwritten character “A” is represented by two strokes (“∧” shape trajectory, “−” shape trajectory) handwritten using the pen 100 or the like. The trajectory of the first “∧” -shaped pen 100 handwritten is sampled in real time, for example, at equal time intervals, thereby obtaining the time-series coordinates SD11, SD12,... SD1n of the “∧” -shaped stroke. Similarly, the trajectory of the “−” shaped pen 100 to be handwritten next is also sampled in real time at equal time intervals, thereby obtaining the time series coordinates SD21, SD22,... SD2n of the “−” shaped stroke.

  The handwritten character “B” is expressed by two strokes handwritten using the pen 100 or the like, that is, two trajectories. The handwritten character “C” is represented by one stroke handwritten using the pen 100 or the like, that is, one locus. The handwritten “arrow” is expressed by two strokes handwritten by using the pen 100 or the like, that is, two trajectories.

  FIG. 4 shows time-series information 200 corresponding to the handwritten character string of FIG. The time series information 200 includes a plurality of stroke data SD1, SD2,. In the time series information 200, the stroke data SD1, SD2,..., SD7 are arranged in time series in the order of handwriting, that is, the order in which a plurality of strokes are handwritten.

  In the time series information 200, the first two stroke data SD1 and SD2 indicate two strokes of the handwritten character “A”, respectively. The third and fourth stroke data SD3 and SD4 indicate two strokes constituting the handwritten character “B”, respectively. The fifth stroke data SD5 indicates one stroke constituting the handwritten character “C”. The sixth and seventh stroke data SD6 and SD7 indicate two strokes constituting the handwritten “arrow”, respectively.

  Each stroke data includes a plurality of coordinates corresponding to a plurality of points on the trajectory of one stroke. In each stroke data, a plurality of coordinates are arranged in time series in the order in which the strokes are written. For example, for the handwritten character “A”, the stroke data SD1 is a coordinate data series (time series coordinates) corresponding to each point on the locus of the stroke of the “∧” shape of the handwritten character “A”, that is, n coordinates. Data SD11, SD12,... SD1n are included. The stroke data SD2 includes coordinate data series corresponding to each point on the trajectory of the stroke of the “−” shape of the handwritten character “A”, that is, n pieces of coordinate data SD21, SD22,. Note that the number of coordinate data may be different for each stroke data.

  Each coordinate data indicates an X coordinate and a Y coordinate corresponding to a certain point in the corresponding locus. For example, the coordinate data SD11 indicates the X coordinate (X11) and the Y coordinate (Y11) of the start point of the “∧” -shaped stroke. SD1n indicates the X coordinate (X1n) and Y coordinate (Y1n) of the end point of the “∧” -shaped stroke.

  Further, each coordinate data may include time stamp information T corresponding to the time when a point corresponding to the coordinate is handwritten. The handwritten time may be either absolute time (for example, year / month / day / hour / minute / second) or relative time based on a certain time. For example, the absolute time (for example, year / month / day / hour / minute / second) when the stroke is started is added to each stroke data as time stamp information, and each coordinate data in the stroke data indicates a difference from the absolute time. The relative time may be added as time stamp information T.

  Furthermore, information (Z) indicating writing pressure may be added to each coordinate data.

FIG. 5 shows a system configuration of the tablet computer 10.
As shown in FIG. 5, the tablet computer 10 includes a CPU 101, a system controller 102, a main memory 103, a graphics controller 104, a BIOS-ROM 105, a nonvolatile memory 106, a wireless communication device 107, an embedded controller (EC) 108, and the like. .

  The CPU 101 is a processor that controls operations of various components in the tablet computer 10. The processor includes processing circuitry. The CPU 101 executes various computer programs loaded into the main memory 103 from the nonvolatile memory 106 that is a storage device. These programs include an operating system (OS) 201 and various application programs. The application program includes a handwritten note application program 202. This handwritten note application program 202 is a digital notebook application capable of taking notes. The handwritten note application program 202 has a function for creating and displaying the above-mentioned handwritten document, a function for editing a handwritten document, an automatic complementing function, and the like.

  The CPU 101 also executes a basic input / output system (BIOS) stored in the BIOS-ROM 105. The BIOS is a program for hardware control.

  The system controller 102 is a device that connects the local bus of the CPU 101 and various components. The system controller 102 also includes a memory controller that controls access to the main memory 103. The system controller 102 also has a function of executing communication with the graphics controller 104 via a PCI Express standard serial bus or the like.

  The graphics controller 104 is a display controller that controls the LCD 17 </ b> A used as a display monitor of the tablet computer 10. The display controller includes a display control circuit. The touch screen display 17 including the LCD 17A (the screen of the touch screen display 17 / LCD 17A) has a rectangular shape including two long sides and two short sides.

  A display signal generated by the graphics controller 104 is sent to the LCD 17A. The LCD 17A displays a screen image based on the display signal. A touch panel 17B is disposed on the LCD 17A.

  The wireless communication device 107 is a device configured to perform wireless communication such as wireless LAN or 3G mobile communication. The EC 108 is a one-chip microcomputer including an embedded controller for power management. The EC 108 has a function of turning on or off the tablet computer 10 in accordance with the operation of the power button by the user.

  The tablet computer 10 may include a peripheral interface for communicating with the input device 109 (mouse, keyboard, etc.).

<Characteristics of handwritten note application program 202>
Hereinafter, some features of the handwritten note application program 202 will be described.

<Basic features of application>
The handwritten note application program 202 automatically and periodically saves the page being edited. When the handwritten note application program 202 is stopped / terminated, the handwritten note application program 202 automatically saves the page being edited.

  The handwritten note application program 202 supports a plurality of languages. The handwritten note application program 202 uses a language set on the OS setting.

  The handwritten note application program 202 includes a character recognition engine. This character recognition engine supports Japanese, English, Korean, Russian, Chinese and various other languages.

<Auto completion function>
The handwritten note application program 202 has an automatic complement function. The auto-completion function displays a list of handwriting candidates that are input candidates determined from strokes (provisional strokes) input by handwriting on the screen. The temporary stroke means a stroke to be automatically complemented, that is, a stroke input in the automatic complement mode. When any one of the handwriting candidates is selected by the user, the automatic completion function inputs the selected handwriting candidate instead of the temporary stroke.

1. User turns on auto-completion mode (user interface (UI) can switch auto-completion mode on and off)
2. The user inputs a stroke (temporary stroke) on the screen. This stroke is displayed on the screen.

  3. The handwriting note application program 202 determines some handwriting candidates and creates a list of handwriting candidates (handwriting candidate menu). Each handwriting candidate is a stroke sequence including one or more strokes. The handwriting candidate may be a handwriting that includes a handwriting portion similar to the provisional stroke at the head portion thereof.

  4). The handwriting note application program 202 displays a list of handwriting candidates (handwriting candidate menu) on the screen. For example, when a temporary stroke corresponding to the character “a” is input by the user, a handwriting candidate menu including a handwriting corresponding to the character string “apple” and a handwriting corresponding to the character string “application” is displayed on the display. Also good.

  5. When the user selects one handwriting candidate on the handwriting candidate menu, the handwriting note application program 202 inputs this handwriting candidate instead of the temporary stroke. That is, the provisional stroke input by the user is complemented using the selected handwriting candidate. This handwriting candidate is a finalized (entered) handwriting. The size (height, character spacing) of the handwriting candidate may be automatically adjusted to a size corresponding to the size of the stroke input before the temporary stroke, for example.

  In the auto-completion function, handwriting candidates are extracted from handwritten information input in the past by the user. This means that the user can reuse the handwritten information input by the user in the past. A set of handwriting collected from handwritten information input in the past by the user may be stored in the auto-complementary dictionary data database. The handwritten note application program 202 can automatically collect a set of handwritings from the user's handwritten notebook, and store the handwriting sets in the auto-complementary dictionary data database. The handwritten note application program 202 can acquire one or more handwriting candidates determined from the temporary strokes from the automatic complement data database. The user can also invalidate any handwriting candidate in the auto-completion data database. The invalid handwriting candidate is not used for the automatic completion process.

  The handwriting note application program 202 displays a handwriting candidate list box as a handwriting candidate menu in order to present more handwriting candidates to the user. The handwriting candidate list box may include a plurality of columns, for example, three columns. In each example, for example, three handwriting candidates may be displayed. Each column contains three button areas. Three handwriting candidates are respectively displayed in the three button areas of each column. Each button area is an operation area for selecting a corresponding handwriting candidate.

  When the user taps an operation area corresponding to a handwriting candidate (for example, a button area corresponding to a handwriting candidate), the handwriting candidate corresponding to the operation area is entered (input) on the page. The handwriting candidate list box may further display a confirmation button.

  When the user taps the confirm button, the handwritten note application program 202 enters (inputs) the temporary stroke on the page.

  The display position of the handwriting candidate list box will be described below.

  The display position of the handwriting candidate list box is preferably placed at a position where the user can easily operate the handwritten input of the temporary stroke. This is because the user can easily select a desired handwriting candidate by moving the pen or finger slightly. The handwritten note application program 202 displays a handwriting candidate list box at a position corresponding to the temporary stroke input position, for example, above the temporary stroke input position.

  The handwritten note application program 202 can accept left-handed or right-handed settings. The user can select the left-handed mode or the right-handed mode with the setting tool provided by the handwriting note application program 202.

  Alternatively, the setting tool of the OS 201 may set a left-handed mode or a right-handed mode according to a user operation.

  The handwriting note application program 202 can change the layout of the handwriting candidate list box according to the left-handed mode / right-handed mode.

  Next, the configuration of the handwritten note application program 202 will be described with reference to FIG.

  The handwritten note application program 202 includes a pen locus display processing unit 301, a time-series information generation unit 302, an editing processing unit 303, a page storage processing unit 304, a page acquisition processing unit 305, a handwritten document display processing unit 306, a processing unit 308, and the like. Is provided.

  The handwritten note application program 202 creates, displays, and edits a handwritten document (handwritten data) by using stroke data input using the touch screen display 17. The touch screen display 17 is configured to detect the occurrence of events such as “touch”, “move (slide)”, and “release”. “Touch” is an event indicating that an external object has touched the screen. “Move (slide)” is an event indicating that the contact position has been moved while an external object is in contact with the screen. “Release” is an event indicating that an external object has been released from the screen.

  The pen locus display processing unit 301 and the time-series information generation unit 302 receive a “touch” or “move (slide)” event generated by the touch screen display 17 and thereby detect a handwriting input operation. The “touch” event includes the coordinates of the contact position. The “movement (slide)” event also includes the coordinates of the contact position of the movement destination. Therefore, the pen locus display processing unit 301 and the time-series information generation unit 302 can receive a coordinate sequence corresponding to the movement locus of the touch position from the touch screen display 17.

  The pen locus display processing unit 301 functions as a display control unit configured to display a handwriting (one or more strokes) input by handwriting on the screen of the touch screen display 17. The pen locus display processing unit 301 receives a coordinate sequence from the touch screen display 17. Based on this coordinate sequence, the pen locus display processing unit 301 displays a plurality of strokes input by a handwriting input operation using the pen 100 or the like on the screen of the LCD 17 </ b> A in the touch screen display 17. The pen locus display processing unit 301 can execute various display controls related to UI display under the control of the processing unit 308.

  The time series information generation unit 302 receives the above-described coordinate sequence output from the touch screen display 17. Based on this coordinate sequence, the time-series information generating unit 302 generates a plurality of stroke data (time-series information) corresponding to the above-described plurality of strokes. These stroke data, that is, the coordinates corresponding to each point of each stroke and the time stamp information of each stroke may be temporarily stored in the work memory 401.

  The page storage processing unit 304 stores handwritten document information including a plurality of stroke data corresponding to a plurality of strokes in a handwritten note database 402 </ b> A in the storage medium 402. As described above, the storage medium 402 may be any of a storage device in the tablet computer 10, a storage device in the personal computer 1, and a storage device in the server 2.

  The page acquisition processing unit 305 reads arbitrary handwritten document information from the storage medium 402. The read handwritten document information is sent to the handwritten document display processing unit 306. The handwritten document display processing unit 306 analyzes the handwritten document information, and displays a plurality of strokes indicated by the plurality of stroke data in the handwritten document information as pages (handwritten pages) based on the analysis result.

  The edit processing unit 303 executes a process for editing the currently displayed page. That is, the edit processing unit 303 executes an edit process for deleting or moving one or more strokes in a plurality of displayed strokes according to an edit operation performed by the user. Further, the edit processing unit 303 updates the handwritten document in order to reflect the result of the editing process on the displayed handwritten document.

  The user can delete an arbitrary stroke in a plurality of displayed strokes by using an “eraser” tool or the like. Further, the user can select an arbitrary part in the displayed handwritten page by using a “range selection” tool for enclosing an arbitrary part on the screen with a rectangle or a free frame.

  The processing unit 308 can execute various processes including the above-described automatic complement. The processing unit 308 includes, for example, an automatic complement processing unit 308A, a box display control unit 308B, and the like.

  The automatic complement processing unit 308A is a processing unit configured to execute the above-described automatic complement function. In the auto-completion mode, the auto-complementation processing unit 308A predicts a handwriting that the user intends to hand-write, that is, a stroke string (handwritten character string), based on the provisional stroke input by handwriting and the handwritten document information. Then, the automatic complement processing unit 308A displays a plurality of handwriting candidates determined from the temporary stroke on the screen of the LCD 17A. In the display area of the plurality of handwriting candidates (the above-mentioned handwriting candidate list box), handwriting candidates are arranged according to their priorities.

  For example, when the temporary stroke “a” is input by handwriting, a handwriting candidate such as the handwritten word “add” or “access” may be presented to the user. If the handwritten word “access” is selected by the user, the provisional stroke “a” is complemented by this handwritten word “access”. Then, the handwritten word “access” is entered into the page instead of the temporary stroke “a”. Therefore, the user can easily input the handwriting of the handwritten word “access”.

  When no next handwriting candidate is selected and the next temporary stroke, for example, “b” is input, the auto-completion processing unit 308A next selects a handwriting candidate list determined from the two temporary strokes “ab”. Present in the list box and present to the user.

  The language of the handwritten character string stored in the handwritten document information may be any language. Examples of available languages include English, Japanese, and various other languages. Regarding the English character string, the handwriting (handwritten character string) may be a stroke string corresponding to a block character string or a stroke string corresponding to a cursive character string. A word handwritten in cursive may be composed of one stroke. Therefore, each handwriting (stroke sequence) acquired from the handwritten document information in the automatic complementing process does not necessarily include a plurality of strokes, and may be a single stroke.

  In order to make it possible to easily identify handwriting candidates determined from temporary strokes, the auto-completion processing unit 308A uses a handwriting such as a handwritten character string from a set of strokes (handwritten document information) stored in the handwritten note database 402A. May be automatically collected, and these automatically collected handwritten character strings may be stored in the automatic complement dictionary database 402B.

  In the automatic completion dictionary database 402B, for example, even if a handwritten character (stroke sequence) and a word (character recognition result) corresponding to the handwritten character are stored in meaningful character string units (for example, word units). good. Furthermore, in the automatic complement dictionary database 402B, in addition to a word and a handwriting (stroke sequence), a reading corresponding to the word may be stored. When the word is English, the alias of the word may be stored in the automatic completion dictionary database 402B instead of reading the word.

  First, the auto-completion processing unit 308A may perform character recognition on the temporary stroke input by the user. Then, the automatic completion processing unit 308A refers to the automatic completion dictionary database 402B and finds a word that matches the character recognition result (character string) of the provisional stroke in the forward direction. The automatic complement processing unit 308A acquires the handwriting (stroke sequence) corresponding to the found word from the automatic complement dictionary database 402B as a handwriting candidate determined from the temporary stroke.

  Alternatively, in the automatic completion dictionary database 402B, for example, a handwriting (stroke sequence) and a feature amount corresponding to the handwriting may be stored in units of words. As a feature amount of a handwriting, an arbitrary feature that can represent the handwriting feature of the handwriting can be used. For example, feature amount data representing a stroke shape, stroke stroke direction, stroke inclination, and the like may be used as the feature amount. In this case, the automatic complement processing unit 308A may acquire a handwriting candidate having a feature amount similar to the feature amount of the temporary stroke from the automatic complement dictionary database 402B.

  In the automatic completion dictionary database 402B, for example, additional information corresponding to each handwriting may be registered in units of words. The additional information may include information indicating the appearance frequency (co-occurrence probability) of each handwriting in the handwritten document information. Further, the additional information may include information indicating the use frequency of each handwriting. The usage frequency of each handwriting may be, for example, the number of times each handwriting is selected from the handwriting candidate list box.

  The auto-completion processing unit 308A acquires some handwriting candidates corresponding to the temporary stroke from the auto-complementary dictionary database 402B using at least one of the feature amount of the temporary stroke or the character recognition result of the temporary stroke. Each of these handwriting candidates may be a handwriting having a feature amount similar to the feature amount of the temporary stroke, or may be a handwriting corresponding to a character string that matches forward with the character recognition result of the temporary stroke. Alternatively, each of the handwriting candidates may be a handwriting corresponding to a character string having a feature amount similar to the feature amount of the temporary stroke and matching the character recognition result of the temporary stroke in front.

  The automatic complement processing unit 308A determines the priority (priority order) of each of these handwriting candidates in order to rank these handwriting candidates. In the process for ranking the handwriting candidates, a score may be given to each handwriting candidate. The score of each handwriting candidate may be determined according to the similarity between the feature amount of the provisional stroke and the feature amount of the handwriting candidate. A handwriting candidate having a high similarity is given a high score. The handwriting candidate with the highest score is given the highest priority.

  Or the score of each handwriting candidate may be determined according to the appearance frequency of a handwriting candidate, or the usage frequency of a handwriting candidate. A handwriting candidate having a high appearance frequency is estimated to be a handwriting that is highly likely to be input by the user. For this reason, a high score is given to a handwriting candidate with high appearance frequency. Handwriting candidates that are frequently used are also estimated to be handwriting that is likely to be input by the user. For this reason, a high score may be given to a handwriting candidate with much use frequency (selection frequency).

  Alternatively, the score of each handwriting candidate is (1) the similarity between the feature amount of the temporary stroke and the feature amount of the handwriting candidate, (2) the appearance frequency of the handwriting candidate, and (3) the number of times of selection of the handwriting candidate. May be determined based on

  The automatic complement processing unit 308A can also use the user dictionary database 402C. The user dictionary database 402C stores a large number of handwriting candidates prepared in advance. These handwriting candidates are handwritings respectively corresponding to a large number of words. The user can register a new handwriting in the user dictionary database 402C. Further, the user can delete any handwriting from the user dictionary database 402C.

  The box display control unit 308B controls the display position of the handwriting candidate list box, the layout of the handwriting candidate list box, and the order of handwriting candidates in the handwriting candidate list box. Since handwriting candidates with higher priority (priority order) are more likely to be selected by the user, the box display control unit 308B displays these handwriting candidates in the handwriting candidate list box according to the priority of these handwriting candidates. . That is, the box display control unit 308B controls the display of the handwriting candidate list box in consideration of the handwriting candidate priority and the handwriting candidate display position so that the user can select a candidate smoothly.

  FIG. 7 shows a user interface (editing screen) of the handwritten note application program 202. The handwritten note application program 202 displays a UI (editing screen) shown in FIG. This UI includes four areas: a notebook list 501, a page list 502, an editing toolbar 503, and an editing view area 504.

  A notebook list 501 indicates a list of notebooks managed by the handwritten notebook application program 202. Two icons “Add Note” and “All Pages” near the upper end of the notebook list 501 are command icons. When the “Add Note” icon is tapped by the user, the processing unit 308 executes a process for adding a new node book. When the “All Pages” icon is tapped by the user, the processing unit 308 executes processing for displaying a list of thumbnails corresponding to all pages in all notebooks.

  The “Unclassified pages” icon is a notebook icon indicating a notebook including a page group that does not belong to any notebook of the user. The “Research” icon is a notebook icon indicating the user's notebook (title name “Research”). The “idess” icon is a notebook icon indicating the user's notebook (title name “idess”). The “presentation” icon is a notebook icon indicating the user's notebook (title name “presentation”). The “Minutes” icon is a notebook icon indicating the user's notebook (title name “Minutes”). The “ToDo” icon is a notebook icon indicating the user's notebook (title name “ToDo”).

  These notebook icons can be rearranged by drag and drop.

  A page list 502 shows a list of thumbnails corresponding to pages included in the notebook corresponding to the currently selected notebook icon.

  The top “Add Page” icon in the page list 502 is a command icon. When the “Add Page” icon is tapped by the user, the processing unit 308 executes processing for adding a new page to the notebook being edited.

  A plurality of page icons indicating thumbnails corresponding to a plurality of pages are arranged below the “Add Page” icon. The stroke included in the page corresponding to the selected page icon is displayed in the edit view area 504. The user can display an arbitrary page in the edit view area 504 by selecting an arbitrary page icon in the page list 502.

  These page icons can be rearranged by drag and drop operation. The user can change (customize) the page order by dragging and dropping the page icons.

  The edit toolbar 503 includes a number of buttons for editing the page. The “selection pen” button 511 is used as a “range selection” tool. The user can select one or more objects on the editing view area 504 with this “selection pen”. When the “selection pen” button 511 is tapped by the user, the processing unit 308 displays a menu for changing the selection type (rectangle / free frame / full selection).

  The “eraser pen” button 512 is used as a “eraser” tool. The user can erase one or more strokes on the editing view area 504 with this “eraser pen”. When the “eraser pen” button 512 is tapped by the user, the processing unit 308 displays a menu for changing the eraser size (large / small / entire page).

  A “stroke input pen” button 513 is used to draw a stroke. The user can draw a handwriting (stroke) in the edit view area 504 with the “stroke input pen”. When the “stroke input pen” button 513 is tapped by the user, the processing unit 308 displays a menu showing several preset pens. Each of these preset pens defines, for example, a combination of pen type (fountain pen / pencil / ballpoint pen / pencil / marker / felt pen), width, color, and transparency. The user can select a pen from this menu.

  An “Undo” button 514 is used to cancel the editing operation. A “redo” button 515 is used to redo the editing operation.

  An “automatic completion (recommendation input)” button 516 is a button for turning on or off the automatic completion mode.

  A “camera” button 517 is used to take a photo and import this photo into the edit view area 504. The user can take a picture with the “camera” button. The processing unit 308 activates a capture application program. The capture application program captures an image (photograph) with a camera (Web cam) provided in the tablet computer 10. The processing unit 308 imports the captured image.

  The “touch input mode” button 519 is a button for turning on or off a touch input mode that can be drawn with a finger or a mouse.

  A “help” button 519 is a button for displaying help. The tab button 521 is a button for switching between normal mode / full screen mode. The UI in FIG. 7 corresponds to the normal mode.

  The edit view area 504 is a handwriting input area where handwriting input is possible. The user can input a stroke by hand at a desired location in the edit view area 504. When the user taps the “automatic completion (recommendation input)” button 516, the automatic completion mode turns on. When the automatic complement mode is on, a list of handwriting candidates corresponding to strokes (provisional strokes) input by handwriting is displayed on the edit view area 504 as input candidates.

  FIG. 8 shows an example of a handwriting candidate list box 701 displayed in the edit view area 504.

  If the user inputs a temporary stroke 611 corresponding to the character “a” and temporary strokes 612 and 613 corresponding to the character “p”, the temporary stroke sequence corresponding to the handwritten character string “ap” is displayed in the editing view area 504. Is displayed. Further, a handwriting candidate list box 701 is displayed at the top of the temporary stroke sequence.

  The handwriting candidate list box 701 is composed of a plurality of regions (a plurality of columns). Here, it is assumed that the handwriting candidate list box 701 includes three regions arranged in the horizontal direction, that is, three columns (C1, C2, and C3 in FIG. 9). The leftmost column is displayed at a position determined from the input position of the temporary stroke sequence corresponding to the handwritten character string “ap”, for example, at the top of the temporary stroke sequence. The handwriting candidate list box 701 may be displayed when the temporary stroke 611 corresponding to the handwritten character “a” is input. In this case, the handwriting candidate list box 701 is displayed at a position where the leftmost column is located above the provisional stroke 611.

  In each column of the handwriting candidate list box 701, a plurality of handwriting candidates are arranged in the vertical direction. Here, it is assumed that three candidates are arranged in the vertical direction in each column.

  In the handwriting candidate list box 701, for example, nine handwriting candidates respectively corresponding to nine words “apple”, “apply”, “application”, “appliicate”,... “Approximate” each including the character string “ap”. Is displayed. These handwriting candidates are obtained by searching for handwriting similar to the temporary stroke sequences 611 to 613 from the automatic complement dictionary database 402B. Each handwriting candidate is a stroke sequence including one or more strokes. Each handwriting candidate may be a handwriting including a handwriting similar to the temporary stroke sequence at the head portion thereof. The handwriting candidate list box 701 includes nine button areas. Nine handwriting candidates are displayed in these nine button areas, respectively.

  When the user taps a button area in the handwriting candidate list box 701 with a pen, a handwriting candidate corresponding to the tapped button area is entered (input) on the page being edited. That is, the temporary stroke sequence (one or more strokes) input by the user is complemented using the selected handwriting candidate. This handwriting candidate is a finalized (entered) handwriting. Accordingly, the user can easily input a desired handwriting by handwriting without inputting all the handwriting by handwriting.

  The leftmost column is displayed adjacent to the upper side of the input position of the provisional stroke column, so that the user only shifts the pen tip slightly upward and the three handwritings displayed in the left column are displayed. Any of the candidates can be easily selected. The operation of moving the pen tip in the vertical direction can be performed by moving the fingertip slightly without moving the wrist. In the present embodiment, the top three handwriting candidates with high priority are arranged in the vertical direction in the leftmost column. These top three handwriting candidates are candidates that are highly likely to be selected by the user. Therefore, the user can smoothly select the target handwriting candidate simply by performing an operation of slightly moving the fingertip.

  The handwriting candidate list box 701 may further display a confirmation button 702. When the confirm button 702 is tapped, the temporary stroke is determined as the input (enter) stroke.

  FIG. 9 is an enlarged view of the handwriting candidate list box 701 of FIG.

  In FIG. 9, dotted arrows indicate the priority order of handwriting candidates. The handwriting candidate at the starting point of the arrow, that is, the handwriting candidate corresponding to the handwritten character string “apple” is the handwriting candidate with the highest priority. The handwriting candidate with the second highest priority is the handwritten character string “apply”. The handwriting candidate with the third highest priority is the handwritten character string “application”. The handwriting candidate with the fourth highest priority is the handwritten character string “applicate”. The handwriting candidate with the fifth highest priority is the handwritten character string “append”. The sixth handwriting candidate with the highest priority is the handwritten character string “appendix”. The seventh handwriting candidate with the highest priority is the handwritten character string “approved”. The handwriting candidate with the eighth highest priority is the handwritten character string “appointed”. The handwriting candidate with the ninth highest priority is the handwritten character string “approximate”.

  Thus, the handwriting candidate “apple” having the highest priority is displayed at the position (C1, R1) closest to the temporary stroke input position (current pen position). As described above, less hand movement is required to move the pen in the vertical direction (up and down) than in the horizontal direction (left and right). This is because the operation of moving the pen in the vertical direction can be performed by slightly raising and lowering the fingertip without turning the wrist. For this reason, the handwriting candidate with the second highest priority is displayed at the position (C1, R2) above the handwriting candidate “apple”, not the position (C2, R1) right next to the handwriting candidate “apple”. Yes. The handwriting candidate “application” having the third highest priority is displayed at a position (C1, R3) above the handwriting candidate “apple”.

  If the fourth highest priority handwriting candidate “applicate” is displayed on top of the handwriting candidate “application”, the distance from the current pen position to the display position of the handwriting candidate “applicate” may be too long. There is. That is, in order to select the handwriting candidate “applicate”, there is a possibility that an operation of moving the pen tip greatly in the vertical direction is required.

  For this reason, the handwriting candidate “applicate” having the fourth highest priority is displayed at the position (C2, R1) to the right of the handwriting candidate “apple”. The position (C2, R1) is the lowest position in the column C2 on the right side of the column C1. In order to select the position (C2, R1) with the pen, the user needs to rotate the wrist slightly in the upper right direction. However, this position (C2, R1) is relatively close to the current pen position. Therefore, the position (C2, R1) is a position that is easy to operate next to the position (C1, R3).

  Thus, in this embodiment, the handwriting candidates that are more likely to be selected by the user can be displayed at positions that are easier to select. Therefore, a smooth selection operation of handwriting candidates is possible. Moreover, since the handwriting candidate list box 701 includes a plurality of columns, many handwriting candidates can be presented to the user.

  8 and 9 exemplify the case where the handwriting candidate list box 701 has a table structure of 3 rows × 3 columns, the number of columns in the handwriting candidate list box 701 may be two. The number of lines in the handwriting candidate list box 701 may be two.

  FIG. 10 shows another example of the handwriting candidate list box 701.

  Here, the handwriting candidate list box 701 includes a column C1 and a column C2 arranged in the horizontal direction with the column C1. That is, the handwriting candidate list box 701 has a table structure of 3 rows × 2 columns.

  The handwriting candidate with the highest priority is displayed at a position (C1, R1) in the column C1 in the vicinity of the input position of the temporary stroke. The handwriting candidate with the second highest priority is displayed at the position (C1, R2) in the column C1, which is aligned with the position (C1, R1) in the vertical direction.

  At the position (C2, R1) in the column 2 that is aligned with the position (C1, R1) in the horizontal direction, at least the handwriting candidate with the lower priority than the handwriting candidate with the second highest priority is displayed. For example, as shown in FIG. 10, when the handwriting candidate list box 701 includes three rows, the fourth highest priority handwriting candidate is displayed at the position (C2, R1). If the handwriting candidate list box 701 includes two rows, the handwriting candidate with the third highest priority is displayed at the position (C2, R1).

  When there are only four handwriting candidates similar to the temporary stroke, the positions (C2, R2) and the positions (C2, R3) may be blank.

  FIG. 11 is a diagram for explaining a process of complementing a temporary stroke with a selected handwriting candidate.

  For example, if the handwriting candidate “apple” is tapped with a pen, the handwriting candidate “apple” is input (entered) instead of the temporary stroke sequence 611 to 613. That is, the temporary stroke strings 611 to 613 are replaced with the handwritten character string “apple” 801. The handwriting candidate list box 701 disappears from the edit view area 504.

  If a new temporary stroke is input by handwriting following the entered handwritten character string “apple” 801, a new handwriting candidate list box 701 including a handwriting candidate determined from the new temporary stroke is displayed. . The new handwriting candidate list box 701 is displayed at a position where the left column C1 is positioned on the new temporary stroke.

  If the handwriting candidate list box 701 does not include a desired handwriting, the user handwrites a new temporary stroke at a position subsequent to the temporary stroke sequence 611 to 613. In this case, the display position of the handwriting candidate list box 701 is not changed, and only the handwriting candidates in the handwriting candidate list box 701 are updated. That is, the handwriting candidate in the handwriting candidate list box 701 is updated to a handwriting candidate determined from the temporary stroke sequence 611 to 613 and the new temporary stroke.

  FIG. 12 shows an example in which a handwriting candidate list box 701 is displayed below the provisional stroke input position.

  In the above description, the handwriting candidate list box 701 is displayed above the input position of the temporary stroke. However, when the temporary stroke is input at the top of the page, a display space cannot be secured above the temporary stroke. Therefore, the box display control unit 308B displays the handwriting candidate list box 701 on either the upper side or the lower side of the temporary stroke input position according to the input position of the temporary stroke.

  For example, when a temporary stroke is input in the upper area of the edit view area 504, the box display control unit 308B moves the handwriting candidate list box 701 below the input position of the temporary stroke as shown in FIG. indicate.

  The box display control unit 308B displays the candidates with high priority (priority order) in the downward direction in order from the top of the leftmost column, and when a predetermined number (here, 3) of handwriting candidates is displayed, The continuation of handwriting candidates is displayed in order from the top.

  FIG. 13 is an enlarged view of the handwriting candidate list box 701 of FIG.

  In FIG. 13, dotted arrows indicate the order of priority of handwriting candidates. The handwriting candidate at the starting point of the arrow, that is, the handwriting candidate corresponding to the handwritten character string “apple” is the handwriting candidate with the highest priority. The handwriting candidate with the second highest priority is the handwritten character string “apply”. The handwriting candidate with the third highest priority is the handwritten character string “application”. The handwriting candidate with the fourth highest priority is the handwritten character string “applicate”. The handwriting candidate with the fifth highest priority is the handwritten character string “append”. The sixth handwriting candidate with the highest priority is the handwritten character string “appendix”. The seventh handwriting candidate with the highest priority is the handwritten character string “approved”. The handwriting candidate with the eighth highest priority is the handwritten character string “appointed”. The handwriting candidate with the ninth highest priority is the handwritten character string “approximate”.

  Thus, the handwriting candidate “apple” having the highest priority is displayed at the position (C1, R3) closest to the temporary stroke input position (current pen position). As described above, less hand movement is required to move the pen in the vertical direction (up and down) than in the horizontal direction (left and right). For this reason, the handwriting candidate with the second highest priority is displayed not at the position (C2, R3) on the right side of the handwriting candidate “apple” but at the position (C1, R2) below the handwriting candidate “apple”. Yes. The handwriting candidate “application” having the third highest priority is displayed at a position (C1, R1) below the handwriting candidate “apple”.

  If the fourth highest priority handwriting candidate “applicate” is displayed under the handwriting candidate “application”, the distance from the current pen position to the display position of the handwriting candidate “applicate” may be too long. There is. That is, in order to select the handwriting candidate “applicate”, there is a possibility that an operation of moving the pen tip greatly in the vertical direction is required.

  For this reason, the handwriting candidate “applicate” having the fourth highest priority is displayed at a position (C2, R3) on the right side of the handwriting candidate “apple”. The position (C2, R3) is the uppermost position in the column C2 on the right side of the column C1. In order to select the position (C2, R3) with the pen, the user needs to rotate the wrist slightly in the lower left direction. However, this position (C2, R3) is relatively close to the current pen position. Therefore, the position (C2, R3) is a position that is easy to operate next to the position (C1, R1).

  As described above, in the present embodiment, the box display control unit 308B determines the handwriting candidate in the handwriting candidate list box 701 according to whether the handwriting candidate list box 701 is displayed on the upper side or the lower side of the temporary stroke. Change the order automatically.

  When the handwriting candidate list box 701 is displayed on the upper side of the temporary stroke, the handwriting candidate with the highest priority is displayed below the handwriting candidate with the second highest priority. On the other hand, when the handwriting candidate list box 701 is displayed below the temporary stroke, the handwriting candidate with the highest priority is displayed above the handwriting candidate with the second highest priority.

  As a result, regardless of whether the handwriting candidate list box 701 is displayed on the upper side or the lower side of the temporary stroke, the handwriting candidate that is highly likely to be selected by the user can be displayed at a position that is easier to select.

  FIG. 14 shows the relationship between the structure of the handwriting candidate list box 701 displayed in the right-handed mode and the structure of the handwriting candidate list box 701 'displayed in the left-handed mode.

  The handwriting candidate list box 701 described with reference to FIGS. 9 and 13 has a layout suitable for the right-handed mode. When the right-handed mode is valid, the box display control unit 308B displays a handwriting candidate list box 701 shown in the upper part of FIG. In the handwriting candidate list box 701, the column C2 is located on the right side of the column C1, and the column C3 is located on the right side of the column C2.

  On the other hand, when the left-handed mode is valid, the box display control unit 308B displays a handwriting candidate list box 701 'shown at the bottom of FIG. In the handwriting candidate list box 701 ', the column C2 is positioned on the left side of the column C1, and the column C3 is positioned on the left side of the column C2.

  FIG. 15 shows the relationship between the temporary stroke input position and the display position of the handwriting candidate list box 701 'in the left-handed mode.

  The column C1 of the handwriting candidate list box 701 'is located above the temporary stroke input position. Column C2 is located on the left side of column C1, and column C3 is located on the left side of column C2.

  Therefore, even when the handwriting input operation is performed on the left, the target handwriting candidate can be selected smoothly without largely moving the position of the hand.

  Next, with reference to FIG. 16, a display form at the time of updating the handwriting candidate list box 701 will be described. When a temporary stroke corresponding to the character “w” is input by the user, for example, a handwriting candidate list box 701 including nine handwriting candidates “world”, “web”, “war”. As for the priority, the handwriting candidate “World” is first, “Web” is second, “War” is third, and “when” is ninth. Therefore, the handwriting candidate “World” is displayed at the bottom of the column C1.

  When the user inputs a temporary stroke corresponding to the character “o” after inputting the temporary stroke corresponding to the character “w”, a handwriting candidate is newly determined from these two temporary strokes. Assume that nine handwriting candidates “word”, “world”, “worth”,... “Woe” are determined as new input candidates determined from two temporary strokes. As for the priorities, the handwriting candidate “word” is first, “world” is second, “worth” is third, and “woe” is ninth.

  In this case, normally, the handwriting candidate “world” is displayed in the second entry from the bottom of the column C1 in the handwriting candidate list box 701. However, since the handwriting candidate “World” is the handwriting candidate displayed from the previous time, the user can remember that the handwriting candidate “World” is at the bottom of the column C1 in the handwriting candidate list box 701. There is sex.

  For this reason, in this embodiment, the same handwriting candidate as the handwriting candidate displayed last time is displayed in the same place in the handwriting candidate list box 701 regardless of the priority order of the handwriting candidates in the new input candidates. Is done. That is, the display position in the handwriting candidate list box 701 is not changed for the handwriting candidate “world” common to the previous time and the current time. As a result, the handwriting candidate “world” is displayed at the same display position in the handwriting candidate list box 701 as the previous time, that is, at the bottom of the column C1. The other handwriting candidates “Word”, “Worth”,... Are arranged according to the relationship of these priorities.

  Here, since the previous priority of the previous and current common handwriting candidate “world” was first, the handwriting candidate “world” is displayed at the bottom of the same column C1 as the previous time. If the previous priority of the handwriting candidate “world” is third, the handwriting candidate “world” is displayed at the top of the same column C1 as the previous time.

  Next, a process for automatically changing the starting point of the temporary stroke sequence used for predicting the handwriting candidate while the input of the temporary stroke sequence is continued will be described with reference to FIGS.

  For example, it is assumed that the user intends to input the handwritten character string “a pendant”.

  As shown in FIG. 17, assuming that the temporary stroke 611 corresponding to the character “a” and the temporary strokes 612 and 613 corresponding to the character “p” are input by the pen by handwriting by the user, A handwriting candidate list box 701 is displayed. Note that the handwriting candidate list box 701 may be displayed above the temporary stroke 611 when the temporary stroke 611 corresponding to the character “a” is input.

  In the handwriting candidate list box 701, for example, nine handwriting candidates (handwritten character strings) each starting with “ap” are displayed.

  The handwriting candidate intended by the user does not exist in the handwriting candidate list box 701. Therefore, as shown in FIG. 18, the user inputs a temporary stroke 614 corresponding to the character “e” after the temporary strokes 611 to 613 with a pen without selecting any handwriting candidate. As illustrated in FIG. 18, the automatic complement processing unit 308 </ b> A updates the handwriting candidates in the handwriting candidate list box 701, and writes nine handwriting candidates that are new input candidates determined from the temporary strokes 611 to 613 and the temporary stroke 614. It is displayed in the candidate list box 701. Each of the nine handwriting candidates that are new input candidates is a handwritten character string starting with “ape”. In this case, the display position of the handwriting candidate list box 701 is not changed. That is, the display position of the handwriting candidate list box 701 is fixed at a position where the leftmost column of the handwriting candidate list box 701 is positioned above the temporary stroke 611 (or the temporary strokes 611 and 612). As a result, the handwriting candidates in the handwriting candidate list box 701 can be updated immediately.

  The handwriting candidate intended by the user does not exist in the handwriting candidate list box 701. For this reason, as shown in FIG. 19, the user inputs a temporary stroke 615 corresponding to the character “n” with the pen following the temporary strokes 611 to 614 without selecting any handwriting candidate.

  The automatic complement processing unit 308A recognizes that the input of the temporary stroke is continued without selecting any handwriting candidate. In this case, the auto-complementation processing unit 308A determines the temporary stroke 611 corresponding to the character “a” input first as the input (enter) stroke, and removes the temporary stroke 611 from the temporary strokes 611 to 615. . Then, as illustrated in FIG. 19, the automatic complement processing unit 308 </ b> A displays nine handwriting candidates, which are new input candidates determined from the temporary strokes 612 to 615, in the handwriting candidate list box 701. Each of the nine handwriting candidates that are new input candidates is a handwritten character string starting with “pen”. These nine handwriting candidates include handwriting candidates corresponding to the handwritten character string “pendant”.

  As illustrated in FIG. 20, the user selects a handwriting candidate corresponding to the handwritten character string “pendant” with the pen.

  Then, as illustrated in FIG. 21, the automatic complement processing unit 308 </ b> A edits the handwriting 802 corresponding to the handwritten character string “pendent” in the edit view area 504 instead of the temporary strokes 612 to 615 corresponding to the handwritten character string “pen”. To enter. The handwriting candidate list box 701 disappears from the edit view area 504.

  As described above, the display position of the handwriting candidate list box 701 is not changed while the input of the temporary stroke sequence is continued without selecting any handwriting candidate. For this reason, as the provisional stroke sequence continues to be input, the current pen position moves away from the leftmost column of the handwriting candidate list box 701.

  In the present embodiment, in such a situation, processing for automatically changing the starting point of the temporary stroke sequence is executed. Therefore, there is a high possibility that the handwriting candidate intended by the user can be presented.

  For example, the auto-complementation processing unit 308A may use the provisional stroke sequence (provisional handwriting) on the condition that the ratio of the width (w) to the height (h) of the circumscribed rectangle, that is, w / h exceeds a certain threshold. A process of automatically changing the starting point of the stroke sequence may be executed. Alternatively, the auto-completion processing unit 308A may execute a process of automatically changing the starting point of the temporary stroke sequence on condition that the number of temporary strokes exceeds a certain threshold.

  The flowchart of FIG. 22 shows the procedure of the automatic complement process executed by the CPU 101 under the control of the handwritten note application program 202.

  When the automatic complement mode is on (YES in step S11), the CPU 101 detects the input position on the edit view area 504 of the temporary stroke when the temporary stroke is input by handwriting (step S12). The CPU 101 executes a process for displaying the temporary stroke on the edit view area 504 (step S13). Then, the CPU 101 displays a handwriting candidate list box 701 that is a handwriting candidate display area at a position determined from the temporary stroke input position, for example, at the upper or lower portion of the temporary stroke input position (step S14). In the handwriting candidate list box 701, a list of handwriting candidates determined from temporary strokes is displayed.

  Note that the display position of the handwriting candidate list box 701 may be changed according to the input direction of the handwritten character corresponding to the temporary stroke, that is, according to horizontal writing / vertical writing. In this case, when the input direction of the handwritten character corresponding to the temporary stroke is horizontal, that is, horizontal writing, the handwriting candidate list box 701 may be displayed above or below the input position of the temporary stroke. On the other hand, when the input direction of the handwritten character corresponding to the temporary stroke is the vertical direction, that is, vertical writing, the handwriting candidate list box 701 may be displayed on the right side or the left side of the input position of the temporary stroke, for example. .

  The CPU 101 determines whether or not a provisional stroke is further input by handwriting (step S15). When the provisional stroke is further input by handwriting (YES in step S15), the CPU 101 executes steps S14 to S15 again. In the handwriting candidate list box display process (step S15) when the temporary stroke is further input by handwriting, the CPU 101 does not change the display position of the handwriting candidate list box 701, and the handwriting candidates in the handwriting candidate list box 701 are displayed. To change. A handwriting candidate in the handwriting candidate list box 701 is a set of handwriting candidates determined from a first input temporary stroke and a next input temporary stroke from a set of handwriting candidates determined from the first input temporary stroke. Changed to

  When no further temporary stroke is input by handwriting (NO in step S15), the CPU 101 determines whether or not a handwriting candidate has been selected (step S16). When a handwriting candidate is selected by the user (YES in step S16), the CPU 101 adjusts the size of the selected handwriting candidate (step S17). Then, the CPU 101 complements the temporary stroke using the selected handwriting candidate, and sets the selected handwriting candidate as a handwritten call that has been confirmed (entered) (step S18). In step S18, the selected handwriting candidate is input instead of the temporary stroke.

  If no handwriting candidate is selected (NO in step S16), the CPU 101 determines whether or not the confirm button has been tapped (step S19). When the confirm button has been tapped (YES in step S19), the CPU 101 sets the provisional stroke to the handwriting that has been confirmed (entered) (step S20).

  If the confirmation button is not tapped (NO in step S19), the process returns to step S16 for determining whether a temporary stroke is further input by handwriting.

  The flowchart of FIG. 23 shows the procedure of a handwriting candidate list box display process executed by the CPU 101 under the control of the handwritten note application program 202.

  The CPU 101 determines some handwriting candidates (for example, nine handwriting candidates) from the input one or more provisional strokes (step S31). Then, the CPU 101 displays these handwriting candidates on the screen (step S32). In step S32, these handwriting candidates are displayed in the handwriting candidate list box 701 on the screen according to the priority of these handwriting candidates.

  The candidate handwriting (first handwriting candidate) having the highest priority is displayed at the first position in the handwriting candidate list box 701 determined from the input position of the temporary stroke. For example, when the handwriting candidate list box 701 is displayed at the upper part of the provisional stroke input position, the first handwriting candidate is displayed at the lowest position in the column C1. On the other hand, when the handwriting candidate list box 701 is displayed below the temporary stroke input position, the first handwriting candidate is displayed at the top position in the column C1.

  The candidate handwriting (second handwriting candidate) having the second highest priority is displayed at the second position aligned with the first position in the vertical direction. For example, when the handwriting candidate list box 701 is displayed above the input position of the temporary stroke, the second handwriting candidate is displayed at the second position from the bottom in the column C1. On the other hand, when the handwriting candidate list box 701 is displayed below the input position of the temporary stroke, the second handwriting candidate is displayed at the second position from the top in the column C1.

  And the candidate handwriting (3rd handwriting candidate) which has a priority lower than a 2nd handwriting candidate at least is displayed on the 3rd position side by side with a 1st position. In the case where the top three handwriting candidates with the highest priority are displayed in the column C1, the third handwriting candidate is the handwriting candidate with the fourth highest priority. On the other hand, in the case where the top two handwriting candidates with the highest priority are displayed in the column C1, the third handwriting candidate is the handwriting candidate with the fourth highest priority. When the handwriting candidate list box 701 is displayed at the upper part of the provisional stroke input position, the third handwriting candidate is displayed at the lowest position in the column C2. On the other hand, when the handwriting candidate list box 701 is displayed below the input position of the temporary stroke, the third handwriting candidate is displayed at the top position in the column C2.

  The CPU 101 determines whether or not a provisional stroke is further input by handwriting (step S33). When the provisional stroke is further input by handwriting (YES in step S33), the CPU 101 determines one or more strokes that have already been input and a plurality of handwriting that are new input candidates from the input stroke (step S31). S34).

  The CPU 101 determines whether or not the same handwriting as any of the handwritings previously displayed in the handwriting candidate list box 701 is included in the plurality of handwriting that are new input candidates described above (step S35).

  When the same handwriting as any of the previously displayed handwriting is included in a plurality of handwriting that are candidates for the above-described new input (YES in step S35), the CPU 101 uses the same handwriting as the above-described new input. Regardless of the priority order of the same handwriting in the candidates, the same handwriting is displayed at the same position as the position of the handwritten candidate list box 701 displayed previously (step S36). Each of the other handwritings in the new input candidate is displayed in the remaining position in the handwriting candidate list box 701 according to their priority.

  When the same handwriting as any of the previously displayed handwriting is not included in the plurality of handwriting that is the new input candidate (NO in step S35), the CPU 101 determines that the handwriting is a new input candidate. Are displayed in the handwriting candidate list box 701 according to their priorities (step S37).

  The flowchart of FIG. 24 shows another example of the procedure of the handwriting candidate list box display process.

  In the flowchart of FIG. 24, the processes of steps S41 and S42 are added to the process described in the flowchart of FIG.

  When a provisional stroke is further input by handwriting (YES in step S33), the CPU 101 determines whether a condition for automatically changing the starting point of the temporary stroke sequence (temporary stroke sequence starting point changing condition) is satisfied. (Step S41). Whether or not the temporary stroke sequence starting point changing condition is satisfied is (1) whether the ratio of the width (w) to the height (h) of the circumscribed rectangle of the temporary stroke sequence (temporary handwriting) exceeds a certain threshold value. (2) Whether or not the number of input temporary strokes exceeds a certain threshold value may be determined based on at least one.

  When the temporary stroke sequence starting point changing condition is satisfied (YES in step S41), the CPU 101 changes the starting point of the temporary stroke sequence to be used for predicting the handwriting candidate (step S34). That is, the CPU 101 determines the first temporary stroke in the current temporary stroke sequence as the input (entered) stroke. Then, the CPU 101 determines a plurality of handwriting that are new input candidates from the remaining stroke sequence excluding the first temporary stroke from the current temporary stroke sequence.

  As described above, in the present embodiment, a plurality of handwritings determined from strokes (temporary strokes) input in the automatic complement mode are displayed as input candidates in the display area (handwriting candidate list box). The handwriting candidate list box includes at least a first area (column C1) displayed at a position corresponding to the input position of the temporary stroke, and a second area (column C2) arranged in the horizontal direction with the first area. The highest-priority handwriting (first handwriting) in the plurality of handwritings is displayed at the first position in the first region (column C1) in the vicinity of the temporary stroke input position. The second handwriting with the second highest priority is displayed at the second position in the first region, which is aligned with the first position in the vertical direction. And the 3rd handwriting which is the 3rd handwriting among these handwriting and has a lower priority than the 2nd handwriting at least is arranged in the 2nd field (column C2) in the 2nd field (column C2) horizontally with the 1st position. Displayed at 3 positions.

  Therefore, the first handwriting that is most likely to be selected by the user is displayed at the position that is most easily selected, that is, the first position in the first area (column C1) that is closest to the input position of the temporary stroke. In addition, the second handwriting that is most likely to be selected by the user is not in the above-described third position aligned in the horizontal direction with the first position, but in the above-described second position aligned in the vertical direction with the first position. Is displayed. The operation of moving the pen tip in the vertical direction can be easily performed by moving the fingertip slightly without moving the wrist. Therefore, by displaying the first handwriting and the second handwriting in a state where they are arranged in the vertical direction in the first region, it is possible to display each handwriting that is likely to be selected at a position where it can be easily selected. Therefore, handwriting candidates that are more likely to be selected can be displayed at positions that are easier to select, thereby realizing smooth handwriting input.

  Furthermore, in the present embodiment, when the handwriting candidate list box is displayed at the top of the temporary stroke, the handwriting candidates with high priority are displayed in the upward direction from the bottom of the first area, and a predetermined number is displayed. Then, the continuation of candidates is displayed in order from the bottom of the adjacent second area. When the handwriting candidate list box is displayed at the top of the temporary stroke, the handwriting candidates with high priority are displayed in the downward direction in order from the top of the first area, and the second number adjacent to the second is displayed when a predetermined number is displayed. Candidate continuations are displayed in order from the top of the area.

  Therefore, regardless of whether the handwriting candidate list box is displayed at the upper part or the lower part of the temporary stroke, the handwriting candidate having a high possibility of being selected can be displayed at a position where it can be easily selected.

  Each of the various functions described in this embodiment may be realized by a processing circuit. Examples of processing circuitry include programmed processors, such as a central processing unit (CPU). The processor executes each described function by executing a computer program (instruction group) stored in the memory. The processor may be a microprocessor that includes electrical circuitry. Examples of processing circuits also include digital signal processors (DSPs), application specific integrated circuits (ASICs), microcontrollers, controllers, and other electrical circuit components. Each of the components other than the CPU described in the present embodiment may also be realized by a processing circuit.

  In addition, since various processes of the present embodiment can be realized by a computer program, the computer program can be installed and executed on a computer through a computer-readable storage medium storing the computer program, and Similar effects can be easily realized.

  The CPU in the computer in which this computer program is installed can function as a processor configured to execute the above-described handwriting complementing process. The display controller in the computer can function as a display processor configured to display each stroke on the screen.

  In this embodiment, the case of using a tablet computer has been described as an example. However, the handwritten document processing function of this embodiment can be applied to a normal desktop personal computer. In this case, a tablet or the like that is an input device for handwriting input may be connected to the desktop personal computer.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

Claims (12)

Display control means for displaying the first stroke input by handwriting on the screen;
When the first stroke is input, a process for displaying a plurality of handwriting candidates, which are input candidates determined from the first stroke, on the screen according to the priority of the plurality of handwriting candidates is executed. A handwriting candidate list box of n rows × m columns (n and m are integers of 2 or more), which is a display area of the plurality of handwriting candidates, and a position corresponding to the input position of the first stroke Including m areas including a first area in the first column displayed on the second area and a second area arranged side by side from the first area,
The processing means includes
The highest-priority first handwriting candidate among the plurality of handwriting candidates is displayed at a first position in the first region in the vicinity of the input position of the first stroke in the handwriting candidate list box,
The second highest-priority second handwriting candidate among the plurality of handwriting candidates is displayed at the second position in the first region, which is aligned in the column direction with the first position in the handwriting candidate list box,
The k-th of the plurality of handwriting candidate high priority of the k handwriting candidate (n <k ≦ 2n) are arranged next to next to the first region of the handwriting candidate list box of the second region display at the kth k-th position,
Electronics. The processing means includes
When the right-handed mode is enabled, the second area is displayed on the right side of the first area,
The electronic device according to claim 1, wherein when the left-handed mode is valid, the second area is displayed on the left side of the first area. The processing means includes
When a second stroke is input by handwriting after the input of the first stroke without selecting any of the plurality of handwriting candidates, a new input determined from the first stroke and the second stroke is made. Execute processing for displaying a plurality of handwriting candidates that are candidates in the handwriting candidate list box,
When the same handwriting candidate as one of the handwritings previously displayed in the handwriting candidate list box is included in the plurality of handwriting candidates as the new input candidates, the same handwriting candidate is the new input candidate. 2. The electronic device according to claim 1, wherein the same handwriting candidate is displayed at the same position as the position in the handwriting candidate list box displayed last time regardless of the priority order of the same handwriting candidate. The processing means includes
A new input candidate determined from the first stroke and the second stroke when the second stroke is input by handwriting after the input of the first stroke without selecting any of the plurality of handwriting candidates A process for displaying a plurality of handwriting candidates in the handwriting candidate list box,
When one or more third strokes are input by handwriting after the input of the first stroke and the second stroke without selecting any of the plurality of handwriting candidates that are candidates for the new input, Decide and remove the first stroke as the input stroke,
The electronic device according to claim 1, wherein a plurality of handwriting candidates that are new input candidates determined from the second stroke and the one or more third strokes are displayed in the handwriting candidate list box. Displaying a first stroke input by handwriting on the screen;
Displaying a plurality of handwriting candidates, which are input candidates determined from the first stroke, on the screen according to the priority of the plurality of handwriting candidates when the first stroke is input. The handwriting candidate list box of n rows × m columns (n and m are integers of 2 or more), which is a display area of the plurality of handwriting candidates, is displayed in a first column displayed at a position corresponding to the input position of the first stroke. M regions including the first region and the second region arranged side by side from the first region,
The highest-priority first handwriting candidate among the plurality of handwriting candidates is displayed at a first position in the first region in the vicinity of the input position of the first stroke in the handwriting candidate list box,
The second highest-priority second handwriting candidate among the plurality of handwriting candidates is displayed at the second position in the first region, which is aligned in the column direction with the first position in the handwriting candidate list box,
The k-th of the plurality of handwriting candidate high priority of the k handwriting candidate (n <k ≦ 2n) are arranged next to next to the first region of the handwriting candidate list box of the second region display at the kth k-th position,
Method. When the right-handed mode is enabled, the second area is displayed on the right side of the first area,
6. The method of claim 5, wherein the second area is displayed on the left side of the first area when left-handed mode is enabled. When a second stroke is input by handwriting after the input of the first stroke without selecting any of the plurality of handwriting candidates, a new input determined from the first stroke and the second stroke is made. A plurality of handwriting candidates that are candidates are displayed in the handwriting candidate list box,
When the same handwriting as one of the handwritings previously displayed in the handwriting candidate list box is included in the plurality of handwriting candidates as the new input candidates, the same handwriting candidate is included in the new input candidates. 6. The method according to claim 5, wherein the same handwriting candidate is displayed at the same position as the position in the handwritten candidate list box that was displayed last time, regardless of the priority order of the same handwriting candidate. When a second stroke is input by handwriting after the input of the first stroke without selecting any of the plurality of handwriting candidates, a new input determined from the first stroke and the second stroke is made. A plurality of handwriting candidates that are candidates are displayed in the handwriting candidate list box,
When one or more third strokes are input by handwriting after the input of the first stroke and the second stroke without selecting any of the plurality of handwriting candidates that are candidates for the new input, the first Determine and remove one stroke as the input stroke,
The method according to claim 5, further comprising: displaying a plurality of handwriting candidates that are candidates for new input determined from the second stroke and the one or more third strokes in the handwriting candidate list box. Displaying a first stroke input by handwriting on the screen;
When the first stroke is input, a function of displaying a plurality of handwriting candidates, which are input candidates determined from the first stroke, on the screen according to the priority of the plurality of handwriting candidates is executed. A program to be executed by a computer of an electronic device,
The handwriting candidate list box of n rows × m columns (n and m are integers of 2 or more), which is a display area of the plurality of handwriting candidates, is displayed in a first column displayed at a position corresponding to the input position of the first stroke. M regions including the first region and the second region arranged side by side from the first region,
Displaying a first handwriting candidate with the highest priority among the plurality of handwriting candidates at a first position in the first region in the vicinity of the input position of the first stroke in the handwriting candidate list box; ,
The second highest-priority second handwriting candidate among the plurality of handwriting candidates is displayed at the second position in the first region, which is aligned in the column direction with the first position in the handwriting candidate list box. When,
The k-th of the plurality of handwriting candidate high priority of the k handwriting candidate (n <k ≦ 2n) are arranged next to next to the first region of the handwriting candidate list box of the second region displaying at the kth k-th position;
A program for causing the computer to execute. When the right-handed mode is enabled, the second area is displayed on the right side of the first area,
The program according to claim 9, wherein when the left-handed mode is enabled, the second area is displayed on the left side of the first area. When a second stroke is input by handwriting after the input of the first stroke without selecting any of the plurality of handwriting candidates, a new input determined from the first stroke and the second stroke is made. A procedure for executing a function of displaying a plurality of handwriting candidates as candidates in the handwriting candidate list box;
When the same handwriting candidate as one of the handwritings previously displayed in the handwriting candidate list box is included in the plurality of handwriting candidates as the new input candidates, the same handwriting candidate is the new input candidate. A procedure for displaying the same handwriting candidate at the same position as the position in the handwritten candidate list box displayed last time, regardless of the priority order of the same handwriting candidate in
The program according to claim 9, further comprising: When a second stroke is input by handwriting after the input of the first stroke without selecting any of the plurality of handwriting candidates, a new input determined from the first stroke and the second stroke is made. Displaying a plurality of handwriting candidates that are candidates in the handwriting candidate list box;
When one or more third strokes are input by handwriting after the input of the first stroke and the second stroke without selecting any of the plurality of handwriting candidates that are candidates for the new input, Determining and removing the first stroke as the input stroke;
Displaying a plurality of handwriting candidates, which are new input candidates determined from the second stroke and the one or more third strokes, in the handwriting candidate list box;
The program according to claim 9, further comprising:

Images