JP4552685B2 - Coordinate detection apparatus and coordinate detection method - Google Patents

Description

  The present invention relates to a coordinate detection apparatus and a coordinate detection method for detecting coordinates of a position designated on a display screen of various electronic terminal devices.

  Conventionally, various touch panels have been put to practical use in order to detect the operation position on the display screen of various electronic terminal devices, and in general, many touch panels using a resistive film method or an electrostatic induction method are used ( For example, see Patent Document 1 and Patent Document 2.)

A system using a pen with a camera is known as a technique for detecting coordinates without using a touch panel. This system has a coordinate pattern paper on which a pattern indicating each coordinate is written at each point in a fixed plane area and a coordinate designating pen provided with a camera for photographing the pen tip. When an entry operation is performed using a coordinate designation pen, a surrounding paper image including the pen tip is photographed by a camera, and a designated coordinate position is detected from a coordinate pattern in the photographed image (for example, Patent Documents). 3).
JP 2003-223283 A JP 2002-290553 A Special table 2003-508831 gazette

  However, in the case of a touch panel, there is a problem that the display unit itself becomes thick because a resistive film and a transparent touch panel for voltage induction need to be overlaid on the display screen.

  On the other hand, in the case of the pen system with camera, the coordinates of the position designated by the coordinate designation pen can be accurately detected, but it is necessary to use a dedicated paper, and the coordinates of the operation position on the display screen of the electronic device. Cannot be detected.

  The present invention has been made in view of the above-described problems, and can accurately detect the coordinates of a position directly specifying the display screen without having to overlay a coordinate detection member such as a transparent touch panel on the display screen. An object of the present invention is to provide a coordinate detection apparatus and a coordinate detection method.

A coordinate detection apparatus according to the present invention includes an image reading unit that reads an image at a specified position on a display screen, and divides the display screen into predetermined small areas, and divides a code for specifying each small area into three. An RGB value of display data is generated based on each of the divided values, and each coordinate region is displayed on the display screen by the coordinate information display control means for displaying each small area by the generated RGB value. RGB information is extracted from the image of the small area read by the image reading device under the control of the coordinate information reading control means, the coordinate information reading control means for reading the image of the small area by the image reading means, and the RGB value Coordinate calculating means for calculating a specified position on the display screen by calculating a code for identifying a small area based on

According to the present invention, the display screen is divided into predetermined small areas, the code specifying each small area is divided into three, and RGB values of the display data are generated based on the three divided values. Designation on the display screen by displaying the region with the generated RGB value, extracting the RGB value from the image of the small region read by the image reading device, and calculating the code identifying the small region based on the RGB value Since the calculated position is calculated, it is possible to detect the coordinates of the position directly designated on the display screen without having to overlay a coordinate detection member such as a transparent touch panel on the display screen.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a perspective view showing an external configuration of a coordinate detection apparatus according to an embodiment of the present invention.

  This coordinate detection device detects the coordinates of the designated position by directly specifying the display device 10 using a personal computer (PC) and the display unit (screen) 11 of the display device 10 and outputs the detected coordinates to the display device 10. The pen type reading device 20 is configured. The pen-type reading device 20 is connected to the display device 10 by a communication cable 21.

  FIG. 2 is a perspective view showing a schematic configuration of the pen-type reading device 20 in the coordinate detection device.

  The pen-type reading device 20 includes a pressure detection unit 22 for detecting the contact pressure of the pen tip P that accompanies a pen operation, and a display unit (screen) 11 that is designated for contact by the pen tip P around the pen tip P. The camera unit 23 for taking an image of a certain area is incorporated.

  In this coordinate detection device, when an arbitrary position on the display unit (screen) 11 is designated by the pen-type reading device 20, the display position (for example, 8 × 8 dot display) is displayed on the display unit (screen) 11. Coordinate information (encoded image) having a predetermined display color that is different for each cell) is displayed (see FIG. 3), and an image of coordinate information of a certain area of the display unit (screen) 11 including the designated point of the pen tip P is displayed. The (encoded image) is photographed by the camera unit 23. And based on the image (encoded image) of the coordinate information image | photographed with the camera part 23, the coordinate (cell position) according to the display color of the display cell in which the pen tip P is located is detected. The distance from the predetermined position of the displayed display cell to the pen tip P is calculated by image analysis to obtain the coordinates of the designated point and output to the display device 10.

  Note that coordinate information (encoded image) displayed on the display screen in accordance with the designation operation of the display unit (screen) 11 by the pen-type reading device 20 is only displayed for a moment during which the camera unit 23 performs a photographing operation. , It is not recognized by the user's vision.

  FIG. 3 is a diagram for explaining coordinate information (encoded image) displayed on the display screen in accordance with the designation operation of the display unit (screen) 11 by the pen-type reading device 20 of the coordinate detection device.

  In this embodiment, the display unit (screen) 11 has a size of 1280 × 1024 dots (one dot is composed of three pixels of R, G, and B), and this screen is divided into small regions of 8 × 8 dots ( This division is called a cell), and coordinate information (encoded image) is displayed corresponding to each cell. The coordinate information (encoded image) displayed in each cell is configured such that each of the four corners of 2 × 2 dots is black and the space between them is white so that the image can be individually recognized, and only the lower left corner is 2 × 4. The dot is black and the direction can be recognized. Then, 4 × 4 dots at the center of each cell are set to a predetermined display color having different R, G, and B values depending on the position of the cell on the screen. Then, the coordinates (cells) on the display screen (11) designated by the pen tip P are determined from the display color of the cell where the pen tip P is located in the coordinate information image (encoded image) photographed by the camera unit 23. Position) can be detected.

  FIG. 4 is a block diagram showing a configuration of electronic circuits of the display device (PC) 10 and the pen-type reading device 20 in the coordinate detection device.

  The display device (PC) 10 includes a CPU 12, which controls the operation of each part of the circuit according to a control program stored in advance by the storage device 13.

  In addition to the storage device 13, a RAM 14, a display unit (screen) 11, and a communication IF (interface) unit 15 with the pen-type reading device 20 are connected to the CPU 12.

  The storage device 13 stores a PC control program for controlling the overall operation of the display device (PC) 10 and stores specified points on the display unit (screen) 11 by the pen-type reading device 20. A coordinate detection processing program (main body side) for detecting coordinates and a display control program for displaying a locus corresponding to the coordinates of the designated point detected thereby are stored. The storage device 13 generates coordinate information (encoded image) (see FIG. 3) to be displayed on the display screen (11) in accordance with the coordinate detection process at the time of the pen input operation on the display screen (11). Therefore, a cell color / cell position correspondence table CT in which different R, G, B value data for each cell and coordinate data (Cx, Cy) of each cell position are associated is stored.

  The RAM 14 stores a display data memory 14 a for storing display data to be displayed on the display unit (screen) 11, and coordinate data of a designated point accompanying a pen input operation detected and calculated by the pen-type reading device 20. A working memory required for various PC control processes, such as the coordinate data memory 14b, is secured.

  The pen-type reading device 20 includes a CPU 24, which controls the operation of each part of the circuit according to a control program stored in advance by the ROM 25.

  In addition to the ROM 25, the CPU 24 is connected to a RAM 26, a pressure detection unit 22, a camera unit 23, and a communication IF (interface) unit 27 with the display device (PC) 10.

  The ROM 25 stores a reading device control program for controlling the overall operation of the pen-type reading device 20, and also stores designated points on the display unit (screen) 11 for the display device (PC) 10. A coordinate detection processing program (pen side) for detecting coordinates, a cell position calculation processing program that is a subroutine of the coordinate detection processing program (pen side), a pen position calculation processing program, and the like are stored. Further, in the ROM 25, coordinate information (encoded image) displayed on the display screen (11) of the display device (PC) 10 and photographed by the camera unit 23 in accordance with coordinate detection processing at the time of pen input operation (FIG. 3). In order to specify (calculate) the cell position P (Cx, Cy) where the pen tip P is located from the reference), different R, G, B value data for each cell and the coordinate data (Cx, Cy) of each cell position ) Is associated with the cell color / cell position correspondence table CT.

  In the RAM 26, a pen tip image data memory 26 a for storing image data G of a certain area on the display unit (screen) 11 including the pen tip P photographed by the camera unit 23, a cell position where the pen tip P is located Coordinate calculation data memory 26b for storing calculation work data of P (Cx, Cy) and calculation work data of pen position P (X, Y), designated point coordinates P (X, Y of calculated pen point P A working memory required in accordance with the coordinate detection process, such as a designated point coordinate data memory 26c for storing () is stored.

  Next, the coordinate detection function of the first embodiment by the coordinate detection device having the above configuration will be described.

  FIG. 5 is a flowchart showing handwriting input processing by the coordinate detection apparatus.

  When an arbitrary position on the display unit (screen) 11 of the display device (PC) 10 is designated by the pen tip P of the pen-type reading device 20, the coordinates of the designated point by the pen tip P are detected (step AB). The display of the locus accompanying the movement of the pen tip P on the display screen is executed (step C). By repeating this process, handwriting input for displaying an image of a pen operation locus on the display screen is performed.

  FIG. 6 is a flowchart showing coordinate detection processing (AB) accompanying handwriting input processing by the coordinate detection device, and FIG. 6A is a flowchart showing coordinate detection processing on the display device (PC) 11 side. (B) is a flowchart showing a coordinate detection process on the pen-type reading device 20 side.

  FIG. 7 is a sequence diagram showing a flow (part 1) of coordinate detection processing (AB) executed between the display device (PC) 11 and the pen-type reading device 20 in accordance with the handwriting input processing by the coordinate detection device. is there.

  FIG. 8 is a sequence diagram showing a flow (No. 2) of coordinate detection processing (AB) executed between the display device (PC) 11 and the pen-type reading device 20 in accordance with the handwriting input processing by the coordinate detection device. is there.

  That is, in the coordinate detection process accompanying the handwriting input process by this coordinate detection apparatus, first, the handwriting input to the display unit (screen) 11 is made to the pen-type reading device 20 via the communication IF unit 15 of the display device (PC) 10. In response to this, the presence or absence of contact with the pen tip P is inquired (step A1).

  In the pen-type reading device 20, when the inquiry signal from the display device (PC) 10 is received via the communication IF unit 27 (step B1), the pressure detection unit 22 detects the contact pressure of the pen tip P ( Step B2), based on the detected pressure, it is determined whether or not the pen tip P is in contact with the display unit (screen) 11 (step B3).

  In step B3, when it is determined that the pen tip P is in contact with the display unit (screen) 11, a notification signal “contact” is transmitted to the display device (PC) 10 (step B4).

  When the display device (PC) 10 receives from the pen-type reading device 20 a “contact present” notification signal when the pen tip P contacts the display unit (screen) 11 (step A2), the storage device In order to set the central portion (4 × 4 dots) for each cell (8 × 8 dots) in the display portion (screen) 11 to a different display color according to the cell color / cell position correspondence table CT stored in advance in FIG. Each R, G, B value data is read out and coordinate information (encoded image) of the display section (screen) 11 is generated and displayed (step A3).

  Then, a “display notification” signal indicating that the display of coordinate information has been executed on the display device (PC) 10 is transmitted to the pen-type reading device 20 (step A4).

  When the pen-type reading device 20 receives a “display notification” signal of coordinate information transmitted from the display device (PC) 10 (step B5), the pen 23 centered on the pen tip P is received by the camera unit 23. The coordinate information of a certain area where the tip P is in contact is photographed, and the photographed image data G is stored in the pen tip image data memory 26a in the RAM 26 (step B6).

  Then, a “shooting completion” notification signal of the contact coordinate information image (G) on the display screen (11) of the pen tip P is transmitted to the display device (PC) 10 (step B7).

  When the display device (PC) 10 receives the “shooting completion” notification signal transmitted from the pen-type reading device 20 (step A5), the coordinate information displayed on the display unit (screen) 11 in step A3. Is erased (step A6), and the coordinate data P (X, Y) touched by the pen tip P detected and calculated by the pen-type reading device 20 is put into a reception standby state (step A7).

  In the pen-type reading device 20, the coordinate information of a certain area where the pen tip P is in contact is photographed, and the photographed image data G is stored in the pen tip image data memory 26a in the RAM 26 (step B6). , B7), the cell position P (Cx, Cy) on the display screen (11) is first detected from the image data of the coordinate information centered on the photographed pen tip P, and then the cell position P The coordinates P (X, Y) of the pen tip P in (Cx, Cy) are calculated by image analysis and stored in the designated point coordinate data memory 26c (step D). Then, the data of the designated point coordinates P (X, Y) on the display screen (11) by the pen tip P of the pen type reading device 20 is output to the display device (PC) 10 (step B8).

  When the display device (PC) 10 receives the data of the designated point coordinates P (X, Y) by the pen tip P transmitted from the pen type reading device 20 (step A7), the received pen tip designation is received. The point coordinates P (X, Y) are captured and stored in the coordinate data memory 14b in the RAM 14, and the display point dots on the display unit (screen) 11 corresponding to the designated point coordinates P (X, Y) are input by the pen. Illuminated and displayed as a locus (step A8).

  At this time, confirmation information indicating that the locus display accompanying the detection of the pen input coordinates this time is performed is stored and held in the RAM 14 (step A8 ').

  On the other hand, when it is determined in step B3 in the pen-type reading device 20 that the pen tip P does not touch the display unit (screen) 11, a notification signal “No contact” is sent to the display device (PC) 10. It is transmitted (step B9).

  When the display device (PC) 10 receives a “no contact” notification signal from the pen-type reading device 20 to the display unit (screen) 11 of the pen tip P (step A2 → A9), the current coordinate detection is performed. Confirmation information that the locus display by the processing is not performed is stored and held in the RAM 14.

  Thereby, it is not necessary to superimpose any member for detecting the designated point coordinates on the display unit (screen) 11 of the display device (PC) 10, and the display unit (screen) 11 is pen-shaped. The designated point coordinates P (X, Y) of the pen tip P can be accurately detected and calculated by direct designation operation by the reading device 20, and the pen input locus can be displayed.

  Next, in the coordinate information to be temporarily displayed on the display unit (screen) 11 of the display device (PC) 10 in accordance with the coordinate detection processing, the central part for each cell (8 × 8 dots) of the display unit (screen) 11 A method for setting each R, G, B value data for setting (4 × 4 dots) to different display colors will be described.

  FIG. 9 shows cell setting divisions of coordinate information (encoded image) to be displayed in the coordinate detection process in the display device (PC) 10 of the coordinate detection device and R, G, B value data to be different for each cell. It is a figure for demonstrating the setting method.

  FIG. 10 is a diagram for explaining a rule for setting R, G, B value data from 15-bit data calculated according to the position of each cell (Cx, Cy) in the coordinate information (encoded image). .

  As shown in FIG. 9, when the number of display dots on the display unit (screen) 11 is 1280 × 1024 dots, this display screen is divided into 160 × 128 cells of 8 × 8 dots, and each cell Is represented by (Cx, Cy). For example, the fourth cell in the X direction and the third cell in the Y direction are (Cx (4), Cy (3)).

  The color assigned to a certain cell (Cx, Cy) is calculated by calculating Cx × 128 + Cy, and using the lower 15 bits of the result, the upper, middle and lower 5 bits of the 15 bits are multiplied by 8 respectively. Let G and B be the values.

  Here, the case where the display color of a cell (Cx (4), Cy (3)) is defined concretely is demonstrated.

First, a specific value corresponding to the cell (Cx (4), Cy (3)) is expressed by a predetermined formula “Cx × 128 + Cy”.
4 × 128 + 3 = 515 = 0000 0010 0000 0011 (binary)
And the lower 15 bits of data are used. Here, Cy (3) in the lower 7 bits (000 0011 (= 3 (decimal)))
The upper 8 bits (000 0010 0 (= 4 (decimal)) Cx (4),
It can be seen that

Then, as shown in FIG. 10, 15-bit data (000 0010 0000 0011) corresponding to the cells (Cx (4), Cy (3))
3 data of 5 bits each (0000 00) (10 000) (0 0011)
8 bits data (00000000) (10000000) (00011000) with “000” added to the lower order of each 5-bit data (multiplied by 8)
R value (00000000), G value (10000000), B value (00011000)
Set as.

  With such a setting method, different R, G, B value data can be set for each cell of coordinate information (encoded image) to be displayed in the display device (PC) 10 in accordance with the coordinate detection process.

  The cell color / cell position correspondence table CT stored in the storage device 13 of the display device (PC) 10 and the cell color / cell position correspondence table CT stored in the ROM 25 of the pen-type reading device 20 are set as described above. The coordinate data (Cx, Cy) of each cell position is stored in association with the R, G, B value data that differs for each cell obtained according to the technique.

  That is, in the coordinate calculation process (step D), in order to calculate the cell position P (Cx, Cy) of the pen tip P from the captured image data G of the coordinate information (encoded image) centered on the pen tip P, First, as shown in FIG. 3, the cell at the center of the photographed image G is specified by image analysis of a cell boundary in which the four corners of each cell are black and white, and the specified pen tip P is located. R, G, B values (8 bits each) are extracted from the display color of the cell image, and the 15-bit data is restored and generated from the upper 5 bits of each, and the lower 7 bits are Cy and the upper 8 bits are Cx. The calculated cell position P (Cx, Cy) can be calculated.

  In addition, the setting method of R, G, B value data for displaying each cell in the coordinate information (encoded image) with a different display color is not limited to this. By setting 15-bit cell position data (cell number), which is a serial number to the end cell, and dividing this cell number into three pieces of 5-bit data, the R, G, B value data ( The upper 5 bits may be set.

  FIG. 11 is a diagram showing a general gamma characteristic of a display.

  Since the gamma characteristic of the display unit (screen) 11 in the display device (PC) 10 is as shown in FIG. 11, it is more stable if only the linear halftone is set as the display color of each cell. Thus, the cell position can be specified. In this case, the display of “black” and “white” at the boundary portion of each cell is the maximum / minimum value in the display color of the cell. By doing so, the dynamic range can be set appropriately when an image of coordinate information is captured by the camera unit 23 of the pen tip P, and the cell position can be identified stably.

  FIG. 12 is a flowchart showing the cell position calculation process in the coordinate calculation process (step D) accompanying the coordinate detection process in the pen type reading device 20 of the coordinate detection apparatus.

  When the captured image data G of coordinate information (encoded image) centered on the pen tip P is stored in the pen tip image data memory 26a in the RAM 26, the image data G is read (step D11). The cell at the center of the photographed image G is specified by the image analysis processing of the cell boundary in which the four corners of each cell are black and white between them (step D12).

  Then, 8-bit R, G, and B values are extracted from the display color of the cell image where the pen tip P is located identified in step D12 (step D13) (see FIG. 10), and the R, G, and B values are further extracted. The 15-bit data consisting of the upper 5 bits of each value is restored and generated, and the cell position P (Cx, Cy) where the pen tip P is located is calculated with Cy as the lower 7 bits and Cx as the upper 8 bits. (Step D14).

  Next, the pen point position calculation process in the cell, which is executed after the cell position is determined, will be described.

  FIG. 13 is an example of a photographed image G centered on the position of the pen tip P photographed by the camera unit 13 in accordance with the coordinate detection processing in the pen type reading device 20 of the coordinate detection device. Here, the photographed image of the camera is assumed to be 64 × 64 dots, and the cell position P (Cx, Cy) where the pen tip is located is photographed in a slightly inclined form in the center of the image.

  In FIG. 13, the coordinates (Cx0, Cy0) of the lower left corner at the cell position P (Cx, Cy) obtained according to the cell position calculation process in the coordinate calculation process (step D) are 2 × Two dots are displayed in black, and only the lower left corner is displayed as 2 × 4 dots in black.

  FIG. 14 shows a pen position calculation process (D11 to D14) executed subsequent to the cell position calculation process (D11 to D14) in the coordinate calculation process (step D) accompanying the coordinate detection process in the pen type reading device 20 of the coordinate detection apparatus. It is a flowchart which shows interpolation calculation.

In the pen position calculation process based on this interpolation calculation, the first specified cell is determined based on the vector position relationship between the captured image G centered on the position of the pen tip P and the cell position P (Cx, Cy) as shown in FIG. A vector b _T (= a _T −a ₂ ) up to the position of the pen tip P based on the lower left corner (Cx 0, Cy 0) of P (Cx, Cy), and a vector b x (= a ₁ −a ₂ ) up to the lower right corner ), A vector by (= a ₃ −a ₂ ) up to the upper left is calculated (step D21).

Then, based on the vectors b _T , bx, and by using the lower left corner (Cx0, Cy0) of the cell position P (Cx, Cy) where the pen tip P is located as a reference, the cell position P (Cx, Cy) The position (X ′, Y ′) of the pen tip P is calculated (step D22).

X ′ = (b _T , b _x ) / (b _x , b _x )
Y ′ = (b _T , b _y ) / (b _y , b _y ) (b, b) indicates the inner product of the vectors (X ′, Y ′) calculated in this step D22 are the vectors b _{x and} b _y The length of the coordinate axis is expressed by the number of pixels of the captured image of the camera, and is different from the coordinate value in the actual display image. Since the size of the cell (b _x , b _y ) is 8 × 8 dots on the display screen, in step D23, X ′ and Y ′ are normalized by the length of b _x , b _y , and 8 (dot) Doubled to convert to coordinate values on the display screen.

X = (8X ′) / √ (b _x , b _x )
Y = (8Y ') / √ (b y, b y)
Then, the X and Y values indicating the position of the pen tip P in the cell position P (Cx, Cy) are added to the value obtained by multiplying the values of the cell positions PCx and Cy by 8 times, so that the display unit ( The coordinate value P (X, Y) at which the pen tip P is located on the screen 11 is calculated. The coordinate value P (X, Y) is stored in the designated point coordinate data memory 26c in the RAM 26 and is output to the display device (PC) 10 (step D23 → B8).

  As a result, the pen point designated point coordinate P (X, Y) received by the display device (PC) 11 is captured and stored in the coordinate data memory 14b in the RAM 14, and is stored in the designated point coordinate P (X, Y). The corresponding display point dot on the display unit (screen) 11 is lit and displayed as a pen input locus (step A8).

  Therefore, according to the coordinate detection function of the first embodiment by the coordinate detection device having the above-described configuration, the touch on the display unit (screen) 11 of the display device (PC) 10 is designated by the pen tip P of the pen type reading device 20. And coordinate information (encoded image) in which the display unit (screen) 11 is divided into cells of 8 × 8 dots in accordance with the touch of the pen and different R, G, B data is set for each cell and color-coded. Is displayed, and a certain area of the coordinate information (encoded image) on the display unit (screen) 11 around the designated position of the pen tip P is photographed by the camera unit 23 provided in the pen type reading device 20. Then, the display color of the cell is extracted from the photographed image data G in a certain area including the cell with the pen tip P, and the R, G, B value data is analyzed and the cell position with the pen tip P is analyzed. After P (Cx, Cy) is calculated, the coordinates of the pen tip P in this cell P (Cx, Cy) are calculated by interpolation calculation, and the coordinates designated by the pen tip P on the display unit (screen) 11 are calculated. Since the position P (X, Y) is calculated, it is not necessary to overlay any member for detecting the designated point coordinates on the display unit (screen) 11 of the display device (PC) 10. It is possible to display the pen input locus by accurately specifying and operating the designated point coordinates P (X, Y) of the pen tip P by directly specifying the pen display device 20 on the display unit (screen) 11. .

  In the pen-type reading device 20 in the coordinate detection device of the first embodiment, the designated position of the pen tip P is centered by one camera unit 23 provided along one side surface adjacent to the pen tip P. The image G of the coordinate information (encoded image) of a certain area displayed on the display unit (screen) 11 is captured. In the configuration of the first embodiment, the display unit (screen) using the pen tip P is used. When the designated point on 11 moves in a certain direction of the camera unit 23, the movement locus of the pen tip P displayed according to the designated point coordinate P (X, Y) obtained by the coordinate calculation process (step D) is the camera. As seen from the part 23, the image becomes behind the pen tip P and cannot be photographed. For this reason, as will be described in the second embodiment (see FIGS. 15 and 16), two camera units 23a and 23b are provided on both sides of the pen tip P, and each camera unit 23 is provided. The device housing corresponding to the shooting direction is configured by a transparent window H, and the coordinate locus of the pen point P after the coordinate calculation processing → designated point display processing is determined by a composite image of the images taken by the two camera units 23a and 23b. When there is a difference between the actual pen tip P and the locus display position, the error is calculated, and the specified point coordinate P (X, Y) obtained by the coordinate calculation process is corrected. It is good also as a structure which performs a simple movement locus | trajectory display.

(Second Embodiment)
FIG. 15 is a perspective view showing an external configuration of a pen-type reading device 20A in the coordinate detection device according to the second embodiment of the present invention.

  The pen-type reading device 20A of the second embodiment is provided with two camera units 23a and 23b provided on both sides of the pen tip P, and an apparatus housing corresponding to the shooting direction of each camera unit 23. It is constituted by a transparent window H.

  FIG. 16 is a sequence diagram showing a flow of coordinate detection processing (AB) executed between the display device (PC) 11 and the pen-type reading device 20A in the coordinate detection device according to the second embodiment.

  In the flow of coordinate detection processing (AB) in FIG. 16, the same processing steps as the flow of coordinate detection processing (AB) in FIG. 7 described in the first embodiment are denoted by the same step symbols. Description is omitted.

  That is, the display device (PC) 10 receives the photographing completion signal of the peripheral image G of the pen tip P transmitted from the pen type reading device 20A (step A5), and is displayed on the display unit (screen) 11. When the existing coordinate information (encoded image) is erased (step A6), an erase completion signal for the coordinate information (encoded image) is transmitted to the pen-type reading device 20A (step A6a).

  In the pen type reading device 20A, based on the peripheral image G of the coordinate information (encoded image) photographed around the pen tip P, the specified point coordinates P (X, Y) by the pen tip P is calculated (step D). In response to receiving the erasure completion signal of the coordinate information (encoded image) transmitted from the display device (PC) 11 (step B7a), the locus display of the current designated point coordinates is performed. The peripheral image GA of the pen tip P immediately before is taken and stored in the RAM 26 (step B7b).

  Then, the data of the current designated point coordinates P (X, Y) calculated by the calculation process (step D) of the designated point coordinates P (X, Y) by the pen tip P is transmitted to the display device (PC) 10. Thus (Step B8), the display device (PC) 10 displays the movement locus of the pen tip P at the position of the received designated point coordinate P (X, Y) (Steps A7 and A8). When the locus display completion signal is transmitted to the pen-type reading device 20A (step A8a), the pen-type reading device 20A captures the peripheral image GB of the pen tip P on which the locus display of the current designated point coordinates is performed. Are stored in the RAM 26 (steps B8a and B8b).

  Then, the trajectory display of the current designated point coordinates photographed in step B8b and the peripheral image GA of the pen point P immediately before the trajectory display of the current designated point coordinates photographed in step B7b is performed. Based on the analysis of the difference by comparison with the peripheral image GB of the pen tip P immediately after that, the image position of the pen tip P at the time of calculating the designated point coordinates of the pen tip P, and the latest locus display position corresponding thereto Is detected, and the deviation of the coordinate value is calculated by vector calculation (step B8c) and transmitted to the display device (PC) 10 (step B8d).

  In this case, the direction and unit length of the coordinate axes in the peripheral images GA and GB of the pen tip P are based on the coordinate information (encoded image) on the display unit (screen) 11 previously captured in step B6. Recognition process.

  Thereby, in the display device (PC) 10, the image position of the pen tip P at the time of calculating the designated point coordinates of the pen tip P received from the pen type reading device 20A, and the latest locus display position corresponding to this. The latest locus display of the pen tip P displayed in the step A8 is corrected and redisplayed in accordance with the amount of deviation of the coordinate value from the step (step A8b, A8c), and exactly matches the position designated by the pen tip P. The movement trajectory can be displayed.

  In the coordinate detection apparatus according to the second embodiment, the peripheral image GA of the pen point P immediately before the locus display of the current designated point coordinate is performed and the pen immediately after the locus display of the current designated point coordinate is performed. By comparing with the peripheral image GB of the tip P, the image position of the pen tip P at the time of calculating the specified point coordinates of the pen tip P and the latest locus display position corresponding to this are detected, and the deviation of the coordinate value is detected. However, if high-speed processing is to be performed, from the peripheral image GB based only on the peripheral image GB of the pen tip P immediately after the trajectory display of the current designated point coordinates is performed. It is also possible to calculate and correct the deviation of the coordinate value between the detected image position of the pen tip P and the locus display position.

  Further, in the coordinate detection apparatus of the second embodiment, the coordinate information on the display unit (screen) 11 that has been captured in advance with respect to the direction and unit length of the coordinate axis in each of the peripheral images GA and GB of the pen tip P (reference number) 17), when the peripheral image GB of the pen tip P on which the locus of the designated point coordinates P ′ is displayed is captured as shown in FIG. An index image Q1, Q2, Q3 that is an index of the coordinate axis direction or unit length is additionally displayed on the display unit (screen) 11, and then the peripheral image GB is photographed. The direction of the coordinate axis and the unit length are easy. It is good also as a structure which can be recognized.

  FIG. 17 shows an actual pen additionally displayed along with the locus display of the designated point coordinate P ′ obtained by the coordinate calculation of the pen tip P in the display device (PC) 11 of the coordinate detecting device according to the second embodiment. It is a figure which shows index image Q1, Q2, Q3 for the shift | offset | difference correction | amendment with the front P position.

  That is, the locus display (P ′) is performed on the display unit (screen) 11 of the display device (PC) 10 in accordance with the current designated point coordinates P ′ calculated by the pen type reading device 20A (steps B6 and D). (Step A8), index images Q1, Q2, and Q3 indicating the direction and unit length of the coordinate axes are additionally displayed in the vicinity of the locus display (P ′), which is shown in FIG. As shown in FIG. 8, the pen-type reading device 20A captures a peripheral image GB of the pen tip P for correction of deviation from the actual pen tip P position (step B8b).

  Here, as index indicating the direction and unit length of the coordinate axis in the peripheral image GB of the pen tip P, three index images Q1, Q2 and Q3 are displayed around the coordinate display of the locus (P ′). The three index images Q1, Q2, and Q3 are displayed so that, for example, the length between the three points is determined in advance, and the index Q1 of one point is the origin and the remaining two indices Q2 and Q3 are orthogonal to each other. Let In addition, the two indicators Q2 and Q3 other than the index Q1 at the origin change the display density and display color so that the X-axis direction and the Y-axis direction can be recognized from the index images Q1, Q2 and Q3.

  As a result, as shown in FIG. 17B, the directions X and Y of the coordinate axes and the unit length in the peripheral image GB of the pen tip P can be easily recognized, and the designated point coordinates displayed this time are displayed. The amount of deviation between P ′ and the pen tip P can be calculated in a vector.

  Then, an indicator image for correcting a deviation from the actual pen tip P position additionally displayed in accordance with the locus display of the designated point coordinate P ′ obtained by the coordinate calculation of the pen tip P in the display device (PC) 11. Q1, Q2, and Q3 are erased when the locus is redisplayed in accordance with the deviation correction from the pen tip P position.

  In the coordinate detection device of each embodiment, in the pen-type reading device 20 (20A), the peripheral image G of the pen tip P is captured, and the specified point coordinates P (X, X) of the pen tip P based on the peripheral image G are used. Y), the designated point coordinates P (X, Y) calculated thereby are output to the display device (PC) 10, and the designated point coordinates P (X, Y) are displayed on the display unit (screen) 11. Y) The locus display of the locus is displayed. However, the peripheral image G of the pen tip P photographed by the pen-type reading device 20 (20A) is transmitted to the display device (PC) 10, and the display device (PC) 10 It is good also as a structure which performs the calculation process of the designated point coordinate P (X, Y) of the nib P based on the said peripheral image G. FIG.

  Further, in the coordinate detection device of each of the embodiments, the locus of the designated point coordinate P (X, Y) according to the calculation of the designated point coordinate P (X, Y) of the pen tip P with respect to the display unit (screen) 11. Although the case where the display is performed has been described, it is needless to say that the application control such as menu selection is performed according to the display content of the designated point coordinates P (X, Y).

  The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention when it is practiced. Further, each of the embodiments includes inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in each embodiment or some constituent features are combined, the problems described in the column of the problem to be solved by the invention can be solved. When the effects described in the column of the effect of the invention can be obtained, a configuration in which these constituent elements are deleted or combined can be extracted as an invention.

The perspective view which shows the external appearance structure of the coordinate detection apparatus which concerns on embodiment of this invention. The perspective view which shows the external appearance structure of the pen type reading apparatus 20 in the said coordinate detection apparatus. The figure for demonstrating the coordinate information (coded image) displayed on the said display screen in connection with designation | designated operation of the display part (screen) 11 by the pen type reading apparatus 20 of the said coordinate detection apparatus. FIG. 2 is a block diagram showing a configuration of electronic circuits of a display device (PC) 10 and a pen-type reading device 20 in the coordinate detection device. The flowchart which shows the handwriting input process by the said coordinate detection apparatus. It is a flowchart which shows the coordinate detection process (AB) accompanying the handwriting input process by the said coordinate detection apparatus, The figure (A) is a flowchart which shows the coordinate detection process by the display apparatus (PC) 11 side, The figure (B) is the figure. The flowchart which shows the coordinate detection process by the pen type reading apparatus 20 side. The sequence diagram which shows the flow (the 1) of the coordinate detection process (AB) performed between the display apparatus (PC) 11 and the pen type reading apparatus 20 with the handwriting input process by the said coordinate detection apparatus. The sequence diagram which shows the flow (the 2) of the coordinate detection process (AB) performed between the display apparatus (PC) 11 and the pen type reading apparatus 20 with the handwriting input process by the said coordinate detection apparatus. The cell setting classification of the coordinate information (encoded image) to be displayed in the coordinate detection process in the display device (PC) 10 of the coordinate detection device and the setting method of the R, G, B value data to be different for each cell will be described. Figure to do. The figure for demonstrating the setting rule of the R, G, B value data from 15 bit data calculated according to each cell (Cx, Cy) position of the said coordinate information (coded image). The figure which shows the general gamma characteristic of a display. The flowchart which shows the cell position calculation process in the coordinate calculation process (step D) accompanying the coordinate detection process in the pen type reading apparatus 20 of the said coordinate detection apparatus. The positional relationship between the photographed image G centered on the position of the pen tip P photographed by the camera unit 13 and the cell position P (Cx, Cy) in accordance with the coordinate detection process in the pen type reading device 20 of the coordinate detection device. FIG. A pen position calculation process (interpolation calculation) executed following the cell position calculation process (D11 to D14) in the coordinate calculation process (step D) accompanying the coordinate detection process in the pen type reading device 20 of the coordinate detection apparatus. The flowchart shown. The perspective view which shows the external appearance structure of 20 A of pen-type readers in the coordinate detection apparatus which concerns on 2nd Embodiment of this invention. The sequence diagram which shows the flow of the coordinate detection process (AB) performed between the display apparatus (PC) 11 and the pen type reading apparatus 20A in the coordinate detection apparatus which concerns on the said 2nd Embodiment. The actual pen tip P position additionally displayed in accordance with the locus display of the designated point coordinates P ′ obtained by the coordinate calculation of the pen tip P in the display device (PC) 11 of the coordinate detection device according to the second embodiment. The figure which shows index image Q1, Q2, Q3 for the deviation correction.

Explanation of symbols

10: Display device (PC)
11 ... Display section (screen)
12 ... CPU
13 ... Storage device 14 ... RAM
14a ... Display data memory 14b ... Coordinate data memory 15 ... Communication IF unit 20 ... Pen type reader (first embodiment)
20A ... Pen-type reading device (second embodiment)
21 ... Communication cable 22 ... Pressure detection unit 23 ... Camera unit (first embodiment)
23a, 23b ... Camera unit (second embodiment)
24 ... CPU
25 ... ROM
26 ... RAM
26a ... pen point image data memory 26b ... coordinate calculation data memory 26c ... designated point coordinate data memory 27 ... communication IF unit CT ... cell color / cell position correspondence table P ... pen point G ... peripheral shot image G of the pen point P
P (Cx, Cy): Cell position of the pen tip P P (X, Y): Point coordinates of the pen tip P Q1, Q2, Q3: Index image of coordinate axis direction and unit length

Claims (6)

Image reading means for reading an image at a specified position on the display screen;
The display screen is divided into predetermined small areas, a code for specifying each small area is divided into three, and RGB values of display data are generated based on the respective three divided values, and each small area is generated. Coordinate information display control means for displaying by RGB values ;
Coordinate information reading control means for causing the image reading means to read an image of a small area displayed on the display screen by the coordinate information display control means;
Designated on the display screen by extracting RGB values from the image of the small area read by the image reading device under the control of the coordinate information reading control means and calculating a code for identifying the small area based on the RGB value. Coordinate calculation means for calculating the position ,
A coordinate detection apparatus comprising: The image reading unit is a pen-type reading device, and includes a pressure detection unit that detects a contact pressure of a pen tip with respect to the display screen.
The coordinate information display control means is temporarily divided into predetermined small areas by the coordinate information display control means when the pressure detection unit of the pen-type reading device detects the contact pressure of the pen tip with respect to the display screen. coordinate detecting device according to claim 1, characterized in that to display the image. The image reading means is a pen-type reading device that reads an image on a display screen including the pen tip image,
The coordinate calculation means calculates the specified point coordinates of the pen tip on the display screen based on the image of the small area including the pen tip image read by the coordinate information read control means,
Locus display control means for displaying the locus of the nib designated point on the display screen corresponding to the designated point coordinates of the nib calculated by the coordinate calculating means;
After the trajectory of the pen point designated point is displayed on the display screen by the trajectory display control means, the pen-type peripheral image reading is performed so that the pen-type reading device reads the image on the display screen around the pen point including the pen-point image. Control means;
Calculated by the coordinate calculation means according to the positional relationship between the pen tip image and the display locus of the pen tip designated point included in the display image around the pen tip read by the pen tip peripheral image reading control means. Coordinate correction means for correcting the specified point coordinates of the nib,
Corresponding to the specified point coordinate of the pen tip corrected by the coordinate correcting unit, the locus display control unit corrects and displays the locus of the pen point specified point displayed on the display screen by the locus display control unit. When,
The coordinate detection apparatus according to claim 1, further comprising:   Auxiliary image display control means for displaying an auxiliary image for indicating the coordinate axis direction and unit length of the display screen as the locus display control means displays the locus of the pen point designated point on the display screen. The coordinate detection apparatus according to claim 3, wherein the coordinate detection apparatus is provided. A coordinate detection method for detecting coordinates of a designated position on a display screen of an electronic device controlled by a computer,
The display screen is divided into predetermined small areas, a code for specifying each small area is divided into three, and RGB values of display data are generated based on the respective three divided values, and each small area is generated. Coordinate information display control step to display by RGB values ;
A coordinate information reading control step in which an image of a small area displayed on the display screen in the coordinate information display control step is read by an image reading device that reads an image at a specified position on the display screen;
An RGB value is extracted from the image of the small area read in the coordinate information reading control step , and a specified position on the display screen is calculated by calculating a code for specifying the small area based on the RGB value. A coordinate calculation step;
A coordinate detection method comprising: The image reading device is a pen-type reading device, and includes a pressure detection unit that detects a contact pressure of a pen tip with respect to the display screen,
In the coordinate information display control step, when the pressure detection unit of the pen-type reading device detects the contact pressure of the pen tip with respect to the display screen, the coordinate information display control step temporarily divides into predetermined small areas. coordinate detecting method according to claim 6, characterized in that to display the image.

Images