JP4155059B2 - Electronic blackboard system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention can transmit information drawn on the screen of the electronic blackboard obtained by using the coordinate detection technique to the display device using the communication means, and can virtually reproduce the drawing state of the screen on the display device side. It relates to an electronic blackboard system.
[0002]
[Prior art]
In recent years, the electronic blackboard system uses a coordinate detection technology that automatically detects an operation position on the screen of the electronic blackboard by a sensor, and a trajectory drawn with a marker or the like is a personal computer (hereinafter, referred to as a display device) in real time. An electronic blackboard system having a so-called interactive function in which the screen of the electronic blackboard is virtually reproduced by a control program on the PC can be seen (refer to Patent Document 1, for example).
[0003]
A conventional electronic blackboard system will be described with reference to FIGS.
[0004]
FIG. 8 is a diagram showing a configuration of a conventional electronic blackboard system, and shows a general system in which an electronic blackboard and a PC are connected by a communication cable.
[0005]
An electronic blackboard 81 on which a user draws handwritten characters, a PC 82 that is a display device for displaying the drawn characters and the like, and a communication cable 83 that connects the electronic blackboard 81 and the PC 82 are configured.
[0006]
First, a coordinate detection method necessary for the interactive function will be described. The coordinate detection sensor 85 receives infrared light and ultrasonic waves transmitted simultaneously by the electronic pen 84. The electronic blackboard 81 calculates the distance from the electronic pen 84 to the coordinate detection sensor 85 based on the time difference. Since the coordinate detection sensors 85 are disposed at both ends of the upper part of the screen and the distance between them is known in advance, the electronic blackboard 81 can detect coordinates with the upper left corner of the screen as the origin by trigonometry.
[0007]
Next, the coordinate information detected by the electronic blackboard 81 is sequentially transmitted to the PC 82 via the communication cable 83 in real time. The control program 86 on the PC 82 receives the information and reproduces the contents drawn on the screen from the coordinate information. At this time, the electronic blackboard 81 adds color information to the infrared light emitted from the electronic pen 84, and transmits the color information to the PC 82 simultaneously with the coordinates, so that the control program 86 can reproduce the color.
[0008]
The electronic blackboard 81 has a system in which a plurality of screen surfaces are prepared, the screen is scrolled by pressing a button prepared in the operation unit 87 in FIG. 8, and a new surface appears on the front surface. FIG. 9 is a diagram showing an example in which characters drawn on a conventional electronic blackboard are displayed on the PC, and shows a place where four screens of the electronic blackboard 81 are captured by the control program 86 on the PC.
[0009]
In the control program 86, one screen of the electronic blackboard 81 is handled in units of pages, and a thumbnail window 91 for displaying thumbnails of all pages is used. The control program 86 also displays an image window 92 that is a page corresponding to the surface being drawn on the screen of the electronic blackboard 81, and reproduces the page being drawn on the electronic blackboard 81 using the coordinate information being received. To do.
[0010]
In such an electronic blackboard system, the control program 86 on the PC 82 recognizes the trajectory of handwritten characters drawn on the image board 92, that is, the image window 92, so-called stroke information, and displays the result as a text code character string on the screen. Some are output and can be used to create minutes.
[0011]
The characters shown in FIG. 9 show an example in which characters drawn on the electronic blackboard are displayed on the PC.
[0012]
The results of the recognition processing performed by the control program 86 for the four character strings drawn in the image window 92 are displayed as text code character strings in the right recognition result window 93. Here, each character string is output as a series of text codes separated by a space code, a line feed code, etc., so that simple text editing operations such as spelling correction can be performed after the output.
[0013]
In such a control program 86, when a text cursor is placed on a certain character string or a range is selected with a mouse or a keyboard in order to indicate which stroke information group is recognized by the output text code character string. It is indicated by a rectangular dotted line display or a reverse display that circumscribes a handwritten character that is a stroke information group that is a recognition source. For example, in FIG. 9, when the user selects a range of the character string 95 “Handwriting” from the recognition result window 93, the control program 86 uses the control program 86 to enclose a handwritten character based on the stroke information group. A circumscribed rectangle 94 indicating the area is indicated by dotted lines. As a result, the user can determine that the handwritten character that is the stroke information group from which the character string 95 is recognized is a portion surrounded by the circumscribed rectangle 94.
[0014]
[Patent Document 1]
JP 2000-172422 A (FIG. 1)
[0015]
[Problems to be solved by the invention]
However, in the conventional electronic blackboard system, since there is no search means to know what text code character string is obtained as a result of recognition of a certain stroke information group, the user draws and inputs it on the electronic blackboard There has been a problem that the text code character string of the recognition result cannot be known from the stroke information.
[0016]
Therefore, since it is impossible to search the text code character string displayed in the recognition result window from the handwritten characters displayed in the image window, when the number of characters drawn on the electronic blackboard is relatively small, There is no problem with the search, but when many characters are drawn on one screen of the electronic blackboard, the user needs to visually search for the corresponding text code character string from the recognition result window, which is troublesome. It is complicated.
[0017]
In view of this, the present invention provides an electronic blackboard system that can search a text code character string in a recognition result window subjected to recognition processing from stroke information of handwritten characters drawn on the electronic blackboard.
[0018]
[Means for Solving the Problems]
In order to solve the above problems, an electronic blackboard system of the present invention includes an electronic blackboard that transmits handwritten characters drawn on a screen to the outside, and an image window that directly displays the handwritten characters transmitted from the electronic blackboard as an image, and, an electronic blackboard system and a display device and having a recognition result window displaying the text code string recognition process the handwriting, the electronic blackboard, constitutes a rendered character on the screen Stroke information consisting of coordinate information and color information of a plurality of points on the stroke to be transmitted is transmitted to the display device for one page or more with one screen surface as one page, and the display device is transmitted from the electronic blackboard The stroke information is drawn on the screen by connecting the points indicated by the coordinate information as lines. The handwritten character is displayed in the image window as an image, the handwritten character drawn by the stroke information is recognized by dictionary information, and the recognized text code character string and each recognized text code character A recognition unit list that holds, as recognition unit information, circumscribing rectangular area information that holds the start and end indices in all text code character strings of the column and the rectangular area coordinates surrounding the handwritten character that is the recognition source; , Storing the selected rectangular area information that holds the rectangular area coordinates surrounding the handwritten character indicated by the recognition unit information currently selected by the user as recognition result information, and the text code character string of the recognition result is stored as the recognition result information. recognition result is displayed in the window, optionally user on the image window Select a coordinate Then, the recognition of the recognition one unit information which holds the circumscribed rectangular area information including the coordinates from the result information, retrieves from among the recognized one unit list, retrieved the recognized one unit information held storing the starting index and the range of the end index the recognition result text code string highlighted in the window, the circumscribed rectangular area information in which the recognition one unit information retrieved is held to the selected rectangular area information It is set as the structure which carries out.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
The invention according to claim 1 of the present invention is an electronic blackboard that transmits handwritten characters drawn on a screen to the outside, an image window that displays the handwritten characters transmitted from the electronic blackboard as an image, and the an electronic blackboard system composed of a recognition result display device provided with a window that displays the text code string recognition process of handwritten characters, the electronic blackboard, the strokes constituting the drawn characters on the screen Stroke information consisting of coordinate information and color information of a plurality of points is transmitted to the display device for one page or more with one screen surface as one page, and the display device transmits the stroke information transmitted from the electronic blackboard. By connecting the points indicated by the coordinate information as lines, the handwritten characters drawn on the screen are imprinted. Displayed in the image window as it is, the handwritten characters drawn by the stroke information are recognized by dictionary information, the text code character string of the recognition result, and the entire text of each recognized text code character string A recognition unit list that holds the start and end indexes in the code character string and the circumscribed rectangular area information that holds the rectangular area coordinates surrounding the handwritten character that is the recognition source; The selected rectangular area information holding the rectangular area coordinates surrounding the handwritten character indicated by the selected recognition unit information is stored as recognition result information, and the text code character string of the recognition result is stored in the recognition result window. selecting an arbitrary coordinate display to a user on the image window then, The recognition one unit information which holds the circumscribed rectangular area information including the coordinates from serial recognition result information, retrieves from among the recognized one unit list, the start index the recognition one unit information retrieved is held and The text code character string on the recognition result window in the range of the end index is displayed in reverse video, and the circumscribed rectangular area information held by the searched one recognition unit information is stored in the selected rectangular area information. It is.
[0020]
As a result, since the text code character string of the recognition result can be easily searched from the stroke information side which is the locus of the handwritten character, the user can know the association between the stroke information group and the recognition result quickly and accurately. It is possible to efficiently perform editing work such as making minutes of meetings using electronic blackboards .
[0024]
The invention described in claim 2 is the invention according to claim 1, the 該手writing characters of the image window became recognized source with the retrieved recognized one unit information based on the circumscribed rectangular area information It is an electronic blackboard system that displays a dotted line or reverse so as to enclose it, so that the text code character string of the recognition result can be easily searched from the stroke information side that is the locus of handwritten characters, so that the user can quickly and accurately stroke It becomes possible to know the association between the information group and the recognition result.
[0026]
Hereinafter, embodiments of the present invention will be described with reference to FIGS.
[0027]
8 and the display device of FIG. 9 are the same as those already described in the section of the prior art, and thus description thereof is omitted.
[0028]
(Embodiment 1)
FIG. 1 is a functional block diagram of a PC according to an embodiment of the present invention.
[0029]
In FIG. 1, the PC 82 analyzes the communication data and determines the processing, and the communication means 13 for performing data communication with the input device 11 such as a keyboard and a mouse, the output device 12 such as a CRT, and the electronic blackboard 81. A communication data processing unit 14, a document information storage unit 15 for managing a screen drawing state at an arbitrary time as a page unit, and managing a plurality of pages as a document unit, and a screen on a window based on document information in the document information storage unit 15 It comprises document information drawing means 16 for reproducing the drawing state, and a control unit 17 for controlling the entire processing.
[0030]
FIG. 2A is a diagram showing a specific example of the document information storage unit of the PC according to the embodiment of the present invention. The document information storage unit 15 manages information in units of pages of the electronic blackboard 81 as page information 21 for an arbitrary number of pages. The page information 21 represents page information of one page among a plurality of page information. The page information 21 includes one base image and a stroke information list.
[0031]
The base image is an image that becomes the background of the page, and all strokes are drawn on this image.
[0032]
The stroke information list manages stroke information for reproducing the drawing of each handwritten character in units of strokes written on the electronic blackboard 81 using the electronic pen 84. The stroke information 22 represents one of a plurality of pieces of stroke information, and stores the coordinate data of each character.
[0033]
FIG. 2B is a diagram for showing an example of stroke information of the letter “t” in the English cursive form according to the embodiment of the present invention. As shown in FIG. 2B, “t” is composed of two strokes S1 and S2, and the stroke information 22 is composed of a line width, a line color, and two or more coordinate data. As indicated by black dots in FIG. 2B, S1 stores coordinate data at 10 locations, and S2 stores coordinate data at 3 locations. If the stroke information is stored in this way, the drawing of “t” shown in FIG. 2B can be reproduced in an arbitrary two-dimensional space such as the image window 92 on the PC 82.
[0034]
Specifically, the coordinate data stored in the individual stroke information 22 is connected with lines using the line width and line color in order from the top. Here, if the drawing actually written on the screen with the electronic pen 84 is similar to the drawing to be reproduced, the line may be a straight line or an approximate curve such as a Bezier curve.
[0035]
FIG. 3 is a functional block diagram of the control unit in the embodiment of the present invention. In FIG. 3, the control unit 17 includes a stroke information storage unit 31, a stroke information recognition unit 32, a dictionary information storage unit 33, a recognition result information processing unit 34, a recognition result information storage unit 35, and a text processing unit 36. Consists of
[0036]
The stroke information storage unit 31 stores a stroke information group of the current page.
[0037]
The stroke information recognition unit 32 recognizes the stroke information as a handwritten character, and outputs the text code character string and the stroke information that is the recognition source.
[0038]
The dictionary information storage unit 33 stores dictionary information that is referred to when the stroke information recognition unit 32 recognizes a stroke information group.
[0039]
The recognition result information processing unit 34 receives the text code character string output from the stroke information recognition unit 32 and the stroke information group that is the recognition source, and constructs and processes recognition result information 41 that can be centrally managed.
[0040]
The recognition result information storage unit 35 stores the recognition result information 41 constructed and processed by the recognition result information processing unit 34.
[0041]
The text processing unit 36 acquires the recognition result information 41 stored in the recognition result information storage unit 35 via the recognition result information processing unit 34 or reflects the text operation of the user in the recognition result information 41.
[0042]
FIG. 4 is a diagram showing the configuration of the recognition result information stored in the recognition result information storage unit according to the embodiment of the present invention. The recognition result information 41 includes a recognition code separated by a line feed code for each word. A recognition unit unit list having information of each recognition result as a unit, and selected rectangular region information holding rectangular region coordinates surrounding the handwritten character currently selected by the user. The recognition one unit list holds information of each word on the text code character string as recognition one unit information 42, and the start index and end index on the text code character string in which the recognition results are connected, The rectangular area coordinates that surround the handwritten character that has become are stored as circumscribed rectangular area information.
[0043]
FIG. 5 is a diagram showing a case where information of an image window and a recognition result window is stored as recognition result information in the embodiment of the present invention.
[0044]
In FIG. 5, when the handwritten character displayed in the image window 92 is recognized, the text code character string of the recognition result information 41 becomes the recognition results “Electronic,“ Board ”,“ Handwriting ”,“ Recognition ”, space, Separated by a code such as a line feed and held together.
[0045]
The recognition unit list manages four recognition results as recognition unit information 42. For example, the start index and the end index of the first recognition one-unit information 42 include “0” and “10”. This is because the first recognition result “Electronic” is between the 0th and the 10th of the text. Represents the existence. Although the value of this index may be changed by the user's text editing operation, it is transmitted to the recognition result information processing unit 34 via the text processing unit 36 and updated to the latest information one by one. The circumscribed rectangular area information indicates that the area in the image window 92 of the recognition result “Electronic” of the stroke information group is (110, 50, 510, 130). In this specific example, the area is managed by coordinates (left, top, right, bottom), but any value can be used as long as the area can be understood, such as (left, top, width, height). The selected rectangular area information holds a value of (100, 400, 600, 500), and the user selects the recognition result “Handwriting” managed by the third recognition unit information in the image window 92. Indicates that
[0046]
Next, the flow of processing in the embodiment of the present invention will be described with reference to the flowcharts shown in FIGS.
[0047]
FIG. 6 is a flowchart showing processing during recognition processing according to the present embodiment. Specifically, when the stroke information group stored in the stroke information storage unit 31 is recognized by the stroke information recognition unit 32, it is stored in the recognition result information storage unit 35 by the recognition result information processing unit 34 as the recognition result information 41. Shows the process.
[0048]
First, when the user calls a recognition process for the stroke information storage unit storing the stroke information group currently displayed in the image window 92 by executing a menu or the like, the stroke information recognition unit 32 causes the stroke of the stroke information storage unit to be stored. Information group recognition processing is performed (S61).
[0049]
The stroke information recognition unit 32 determines an optimal recognition result while referring to the dictionary information storage unit 33. When the recognition process is completed, the recognition result information processing unit 34 acquires the number of recognition results of the stroke information group from the stroke information recognition unit 32. That is, the number of words subjected to recognition processing is acquired (S62).
[0050]
Next, the recognition result information processing unit 34 determines whether a recognition result exists (S63).
[0051]
When there is a recognition result, the recognition result information processing unit 34 acquires the recognition result one by one from the stroke information recognition unit 32 (S64).
[0052]
The recognition result information processing unit 34 adds the text code character string that is the recognition result to the text code character string to be connected (S65).
[0053]
Next, the recognition result information processing unit 34 calculates the start and end indexes of the text code character string added earlier, as viewed from the entire character string (S67).
[0054]
Further, the recognition result information processing unit 34 acquires the stroke information that is the recognition source from the stroke information recognition unit 32, and calculates the coordinates (circumscribed rectangular region information) of the rectangular region surrounding the handwritten character (S68).
[0055]
Finally, the recognition result information processing unit 34 adds the text code character string, the start and end indexes, and the coordinates of the circumscribed rectangle to the recognition result information 41 held by the recognition result information storage unit 35 (S69), and further generates In order to determine whether there is a recognition result to be processed, the process proceeds to step S63. In step S63, if there is no remaining recognition result, the process ends.
[0056]
In this way, information for searching for a text code character string as a recognition result is constructed from the stroke information side.
[0057]
FIG. 7 is a flowchart showing processing when searching for a recognition result from the stroke information side according to the embodiment of the present invention.
[0058]
Specifically, the recognition result information processing unit 34 obtains the corresponding recognition unit information from the coordinates specified by the user in the image window 92 from the recognition result information 41 stored in the recognition result information storage unit 35. Indicates the search process.
[0059]
First, the user designates the coordinates of a handwritten character whose user wants to know the recognition result in the image window 92 using the input device 11 such as a mouse. For example, arbitrary coordinates in the circumscribed rectangle 94 are designated. The recognition result information processing unit 34 determines whether an element exists in the recognition unit list of the recognition result information 41 stored in the recognition result information storage unit 35 (S71).
[0060]
If there are no elements in the list, the process ends because there are no more recognition results to be examined. If there is an element in the list, one piece of recognition unit information 42 is acquired (S72), and it is determined whether the coordinate specified by the user exists in the circumscribed rectangular area information held by the recognition unit unit information 42 (S72). S73).
[0061]
If the coordinates do not exist in the circumscribed rectangular area information, the recognition result information processing unit 34 proceeds to step S71 to investigate other elements. If the coordinates exist in the circumscribed rectangle area information, the text processing character string managed by the recognition unit information 42 is the recognition result requested by the user. The selected state is selected according to the range from the index to the end index, and the text which is the recognition result of the stroke information designated by the user with the input device 11 such as the mouse in the image window 92 is extracted (S74).
[0062]
Next, circumscribed rectangular area information of the recognized one unit information 42 searched in S73 is stored in the selected rectangular area information (S75).
[0063]
The circumscribing rectangle of the last selected stroke information group is displayed with a dotted line or reverse display, and the range of this stroke information group is clearly indicated and the process is terminated. In FIG. 9, when an arbitrary coordinate in the circumscribed rectangle 94 is designated, the character string 95 is highlighted (S76).
[0064]
As described above, in the electronic blackboard system according to the present embodiment, since the text code character string of the recognition result can be easily searched from the stroke information, the user can quickly and accurately recognize the handwritten character that is the stroke information and its recognition. You can know the association of results.
[0065]
【The invention's effect】
As described above, according to the electronic blackboard system according to claim 1, the text code character string including the designated coordinates is selected from the text code character strings in the recognition result storage unit according to the coordinates designated in the image window. Because it is extracted, it is possible for the user to know the association between the handwritten characters that have been specified accurately and the recognition result, and the advantageous effect of being able to efficiently perform editing work such as creating minutes of meetings etc. using an electronic blackboard Is obtained .
[0068]
According to the electronic blackboard system of the second aspect of the present invention, the rectangular area surrounding the handwritten character corresponding to the extracted text code character string by the coordinates selected by the user on the image window of the display device. Is displayed, the user can quickly and accurately know the handwritten character of the recognition source.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of a PC in an embodiment of the present invention. FIG. 2 (a) is a diagram showing a specific example of a document information storage unit of the PC in the embodiment of the present invention. FIG. 3 is a functional block diagram of a control unit according to an embodiment of the present invention. FIG. 4 is a diagram for illustrating an example of stroke information of a letter “t” in an English cursive form. The figure which shows the structure of the recognition result information which a result information storage part stores. FIG. 5 is the figure which showed the case where the information of the image window and recognition result window in embodiment of this invention was stored as recognition result information. FIG. 7 is a flowchart showing processing during recognition processing according to the present embodiment. FIG. 7 is a flowchart showing processing when searching for a recognition result from the stroke information side according to the embodiment of the present invention. Figure conventional diagram showing the configuration of an electronic blackboard system [9] Conventional character drawn by the electronic blackboard showing an example in which on the PC [Description of symbols]
11 Input device 12 Output device 13 Communication means 14 Communication data processing unit 15 Document information storage unit 16 Document information drawing unit 17 Control unit 21 Page information 22 Stroke information 31 Stroke information storage unit 32 Stroke information recognition unit 33 Dictionary information storage unit 34 Recognition Result information processing unit 35 Recognition result information storage unit 36 Text processing unit 41 Recognition result information 42 Recognition unit information 81 Electronic blackboard 82 PC
83 Communication cable 84 Electronic pen 85 Coordinate detection sensor 86 Control program 91 Window 92 for displaying thumbnails Image window 93 Recognition result window 94 circumscribed rectangle 95 character string

Claims (2)

An electronic blackboard that transmits handwritten characters drawn on the screen to the outside, an image window that displays the handwritten characters transmitted from the electronic blackboard as an image, and a text code character string that has been processed to recognize the handwritten characters an electronic blackboard system composed of a display device including a recognition result window,
The electronic blackboard
Sending stroke information consisting of coordinate information and color information of a plurality of points on a stroke constituting a character drawn on the screen to one page or more as one page to the display device,
The display device
The handwritten characters drawn on the screen are displayed on the image window as an image by connecting the points indicated by the coordinate information of the stroke information transmitted from the electronic blackboard as lines,
A handwritten character drawn by the stroke information is recognized by dictionary information,
The recognition result text code character string, the start index and the end index in all the text code character strings of each recognized text code character string, and rectangular area coordinates surrounding the handwritten character that is the recognition source A recognition unit list that holds the circumscribed rectangular region information to be held as recognition unit information, and selected rectangular region information that holds the rectangular region coordinates surrounding the handwritten character indicated by the recognition unit information currently selected by the user Is stored as recognition result information,
Displaying the recognition result text code string in the recognition result window;
The user selects an arbitrary coordinate on the image window Then, the recognition one unit information which holds the circumscribed rectangular area information including the coordinates from the recognition result information, retrieves from among the recognized one unit list,
The highlighted text code character string on the recognition result window in the range of the start index and end index held by the searched one unit information of recognition,
The electronic blackboard system, wherein the circumscribed rectangular area information held by the retrieved recognition unit information is stored in the selected rectangular area information . 2. The electronic device according to claim 1 , wherein the handwritten character of the image window that is a recognition source based on the retrieved recognition unit information is displayed in a dotted line or reversely so as to be surrounded based on the circumscribed rectangular area information. Blackboard system.

Images