JP7791064B2 - Display device, pen input system, method, and program - Google Patents

Description

This application relates to a display device, a pen input system, a method, and a program.

Conventionally, there is technology for authenticating users by comparing signature data entered with an electronic pen with one or more stored reference signature data (see, for example, Patent Document 1).

JP 2020-9452 A

However, there is room for improvement in the user authentication methods described above.

In one embodiment, (1) the display device comprises a touch display that accepts input from an electronic pen; a receiving unit that receives from the electronic pen first data corresponding to the detection results of biometric information of a user holding the electronic pen and second data corresponding to the detection results of at least one of acceleration, writing pressure, and angular velocity; and a control unit that, when the first data satisfies a predetermined condition, executes a process to link information about the user to a first image input by the electronic pen based on the second data.

(2) The display device of (1) above is characterized in that, when the first image is input into an application for which a predetermined authority level is set, if the first data satisfies a predetermined condition but the user's authority does not meet the predetermined authority level, the control unit does not perform a process of linking the first image to the user's information, and causes the touch display to display a second image indicating that the first image cannot be input.

(3) In either the display device (1) or (2) above, the control unit is characterized in that it causes the touch display to display the first image and a third image indicating that the first image is linked to user information.

(4) The display device of (3) above is characterized in that, when the touch display detects an input selecting the first image or the third image, the control unit causes the touch display to display a fourth image indicating information about the user.

(5) In any of the display devices (1) to (4) above, the control unit is characterized in that, when the touch display detects an operation to edit the first image, the control unit causes the touch display to display a fifth image prompting input of the first data.

(6) The display device of (5) above is characterized in that the control unit permits editing of the first image if the first data input after the fifth image is displayed satisfies a predetermined condition.

In one embodiment (7), the system comprises an electronic pen and a display device having a touch display that accepts input from the electronic pen, wherein the electronic pen has a first sensor that acquires biometric information of a user holding the electronic pen, at least one second sensor that detects at least one of acceleration, writing pressure, and angular velocity, and a transmitting unit that transmits first data corresponding to the detection result of the first sensor and second data according to the detection result of the at least one second sensor to the display device, and the display device has a receiving unit that receives the first data and the second data and a control unit, and wherein the control unit, when the first data satisfies a predetermined condition, executes a process of linking the user's information to an image input by the electronic pen based on the second data.

In one embodiment, the method (8) is characterized in that, when first data received by a receiving unit from an electronic pen and corresponding to biometric information of a user acquired by the electronic pen satisfies a predetermined condition, the receiving unit executes a process of linking information about the user to an image input to a touch display by the electronic pen based on second data received by the receiving unit from the electronic pen and corresponding to a detection result of at least one of acceleration, writing pressure, and angular velocity applied to the electronic pen.

In one embodiment, (9) the program is characterized in that, when first data received by the receiving unit from the electronic pen and corresponding to the user's biometric information acquired by the electronic pen satisfies a predetermined condition, the program executes a process of linking the user's information to an image input to the touch display by the electronic pen based on second data received by the receiving unit from the electronic pen and corresponding to at least one detection result of the acceleration, writing pressure, and angular velocity applied to the electronic pen.

FIG. 1 is an external view of a display device according to an embodiment. FIG. 2 is a block diagram showing the configuration of a display device according to an embodiment. FIG. 3 is an external view of an electronic pen according to an embodiment. FIG. 4 is a block diagram showing the configuration of an electronic pen according to an embodiment. FIG. 5 is a diagram illustrating an example of the operation of the display device according to the embodiment. FIG. 6 is a diagram showing an example of how an image is drawn on a touch display of a display device according to an embodiment. FIG. 7 is a diagram illustrating an example of a first image according to an embodiment. FIG. 8 is a diagram illustrating an example of a first image according to an embodiment. FIG. 9 is a diagram illustrating an example of the operation of the display device according to the embodiment. FIG. 10 is a diagram showing a display example of the display device according to an embodiment. FIG. 11 is a diagram illustrating an example of the operation of the display device according to the embodiment. FIG. 12 is a diagram illustrating a display example of the display device according to an embodiment. FIG. 13 is a diagram illustrating an example of the operation of the display device according to the embodiment. FIG. 14 is a diagram illustrating a display example of the display device according to an embodiment. FIG. 15 is a diagram illustrating an example of the operation of the display device according to the embodiment. FIG. 16 is a diagram illustrating a display example of a display device according to an embodiment.

Embodiments will be described with reference to the drawings. In the description of the drawings, identical or similar parts are designated by identical or similar reference numerals.

A display device according to one embodiment may be, for example, a terminal such as a smartphone terminal or a tablet terminal. However, the display device is not limited to such terminals and may also be, for example, a personal computer, an electronic whiteboard, or a wearable device such as a head-mounted display.

Figure 1 is an external view of a display device according to one embodiment. The external appearance of the display device 100 will be described with reference to Figure 1.

As shown in FIG. 1, the display device 100 has a housing 101 and a touch display 110.

The touch display 110 is provided with its display surface exposed from the housing 101 of the display device 100. The touch display 110 has a touch panel 111 and a display unit 112. The display unit 112 is provided so as to overlap the touch panel 111, and the display area of the display unit 112 overlaps with the touch panel 111. However, instead of the display unit 112 and the touch panel 111 being provided so as to overlap each other, the display unit 112 and the touch panel 111 may be provided as separate entities.

Figure 2 is a block diagram showing the configuration of a display device according to one embodiment. The functional configuration of the display device 100 will be described with reference to Figure 2.

As shown in FIG. 2, the display device 100 has a touch display 110 (touch panel 111 and display unit 112), a communication interface 120, a memory unit 130, and a control unit 140.

The touch panel 111 detects position information (hereinafter simply referred to as position information of the electronic pen, etc.) indicating the position on the touch panel 111 where an operation is performed with a pointer such as the electronic pen 200 described below, and transmits the detected position information of the electronic pen, etc. to the control unit 140. The detection method of the touch panel 111 may be, for example, a pressure-sensitive method, a capacitance method, or an electromagnetic induction method, but is not limited to these.

The display unit 112 displays objects such as characters, images, or figures (hereinafter simply referred to as objects) on the screen based on signals input from the control unit 140. The control unit 140 can also display objects on the display unit 112 based on position information of an electronic pen or the like received from the touch panel 111. In other words, the display unit 112 can display objects handwritten on the touch display 110 using an electronic pen 200 or the like. The display unit 112 is configured with a display panel such as a liquid crystal display (LCD), an electroluminescence display (EL display), or a MEMS (Micro Electro Mechanical System) shutter display.

The communication interface 120 can communicate with other devices via a wireless connection. Wireless communication standards supported by the communication interface 120 include, for example, cellular communication standards such as 2G, 3G, 4G, and 5G, and short-range wireless communication standards. Examples of short-range wireless communication standards include IEEE 802.11, Bluetooth (registered trademark), IrDA (Infrared Data Association), and NFC (Near Field Communication).
Examples of WPAN communication standards include Wireless Field Communication (WFP) and Wireless Personal Area Network (WPAN). An example of a WPAN communication standard is ZigBee (registered trademark). The communication interface 120 may communicate with other devices via a wired connection. When the communication interface 120 is connected via a wire, it may be an interface that complies with a wired communication standard such as Universal Serial Bus (USB).

The communication interface 120 functions as a receiving unit that receives data from the electronic pen 200, and transmits the data received from the electronic pen 200 to the control unit 140.

The storage unit 130 includes at least one memory that stores the program 131, data, etc. The storage unit 130 is also used as a working area that temporarily stores the processing results of the control unit 140. The storage unit 130 may be configured to include any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium. The storage unit 130 may be configured to include multiple types of storage media. The storage unit 130 may be configured to include a combination of a portable storage medium such as a memory card, an optical disk, or a magneto-optical disk, and a storage medium reader. The storage unit 130 includes RAM (Random Access Memory).
The storage unit 130 may be configured to include a storage device used as a temporary storage area such as a USB flash drive (USB Flash Memory). The storage unit 130 may, for example, store biometric information of a user in advance. The biometric information may include, for example, at least one of fingerprint information, a facial image, and voiceprint information. The storage unit 130 may further store user information. The user information may include, for example, at least one of a user ID (an identifier indicating a user), a facial image of the user, the user's name, the user's affiliation, the user's biometric information, and the user's job title. The storage unit 130 may, for example, store information indicating user authority. The information indicating user authority is, for example, information corresponding to the editing authority level of a predetermined application and indicating authority pre-associated with the user (hereinafter also referred to as user authority information).

Program 131 provides various functions, such as the ability to perform biometric authentication of a user and the ability to link user information to an object, based on the control of devices that make up the display device 100, such as the control unit 140.

The control unit 140 is a processing unit. Examples of processing units include, but are not limited to, a CPU (Central Processing Unit), a SoC (System-on-Chip), an MCU (Micro Control Unit), an FPGA (Field-Programmable Gate Array), and a core processor. The control unit 140 also includes a GPU (Graphics Processing Unit), a VRAM (Video RAM), etc., and displays various images on the display unit 112. The control unit 140 may be composed of multiple processing units, and various controls may be performed through the cooperation of these multiple processing units.

The control unit 140 comprehensively controls the operation of the display device 100 to realize various functions. The control unit 140 performs various operations based on input detected by the touch panel 111. The control unit 140 may output according to the input signal via the communication interface 120, the display unit 112, etc.

By executing the control program 131, the control unit 140 realizes the functions provided by the control program 131 as processing by the control unit 140.

The control unit 140 may detect whether the display device 100 is connected to the electronic pen 200 via the communication interface 120.

When the display device 100 and the electronic pen 200 are connected via the communication interface 120, the control unit 140 may control the touch panel 111 so that it can accept position information from the electronic pen, etc. In other words, the control unit 140 may control the touch panel 111 so that it does not accept input from the electronic pen 200 until the display device 100 and the electronic pen 200 are connected via the communication interface 120.

The electronic pen 200 is described below.

Figure 3 is an external view of an electronic pen according to one embodiment. The external appearance of the electronic pen 200 will be described with reference to Figure 3.

As shown in FIG. 3, the electronic pen 200 has a housing 201, a core body 202, and a first sensor 210.

The housing 201 is cylindrical. The core body 202 is provided at the bottom end of the electronic pen 200 (housing 201). The first sensor 210 is provided at the bottom end of the electronic pen 200 (housing 201), but may be provided at an appropriate position, such as at the top end. When using the electronic pen 200 to handwrite an object on the touch display 110, the user grasps the housing 201, brings the core body 202 into contact with the touch display 110, and moves the electronic pen 200 over the touch display 110 to draw the desired object.

Figure 4 is a block diagram showing the configuration of an electronic pen according to one embodiment. The functional configuration of the electronic pen 200 will be described with reference to Figure 4.

As shown in FIG. 4, the electronic pen 200 has a first sensor 210, a second sensor 220, a communication interface 230, a memory unit 240, and a control unit 250.

The first sensor 210 is a biometric sensor. The first sensor 210 acquires biometric information of the user holding the electronic pen 200 and outputs data corresponding to the acquired biometric information of the user to the control unit 250. The first sensor 210 may be, for example, a fingerprint sensor that acquires the user's fingerprint data, a camera that acquires facial image data including at least a portion of the user's face, a microphone that acquires the user's voice data, or one or more sensors that combine these sensors. The first sensor 210 is capable of acquiring at least one of the aforementioned fingerprint data, facial image data, and voice data as the acquired biometric information of the user.

The second sensor 220 is a sensor for estimating the amount of physical change that occurs in the electronic pen 200 when a user uses the electronic pen 200 to handwrite an object on the touch display 110. The second sensor 220 is, for example, an acceleration sensor, an angular velocity sensor, a pen pressure sensor configured to detect the pressure applied to the core body 202, or one or more sensors that combine these sensors. The second sensor 220 transmits the acquired data to the control unit 250.

The communication interface 230 communicates wirelessly or via a wired connection. The communication standard supported by the communication interface 230 can be, for example, the cellular communication standard, short-range wireless communication standard, or wired communication standard described for the communication interface 120. The communication interface 230 functions as a transmitter that transmits data acquired by the electronic pen 200 (e.g., data acquired by the first sensor 210 and the second sensor 220) to the display device 100.

The storage unit 240 includes at least one memory that stores programs and data. The storage unit 240 is also used as a working area for temporarily storing processing results of the control unit 250. The storage unit 240 may be configured to include any non-transitory storage medium, such as a semiconductor storage medium or a magnetic storage medium. The storage unit 240 may be configured to include multiple types of storage media. The storage unit 240 may be configured to include a storage device used as a temporary storage area, such as RAM. The storage unit 240 may, for example, store the user's biometric information in advance. The biometric information includes, for example, at least one of fingerprint information, facial image, and voiceprint information.

The control unit 250 is a processing unit. Examples of the processing unit include, but are not limited to, a CPU, an SoC, an MCU, an FPGA, or a core processor.
50 may be composed of a plurality of arithmetic processing units, and various controls may be executed by the cooperation of the plurality of arithmetic processing units.

The control unit 250 may detect whether the electronic pen 200 is connected to the display device 100 via the communication interface 230. When the electronic pen 200 is connected to the display device 100 via the communication interface 230, the control unit 250 may control the electronic pen 200 to transmit data acquired by the first sensor 210 and the second sensor 220 to the display device 100. In other words, the control unit 250 may control the electronic pen 200 not to transmit data acquired by the first sensor 210 and the second sensor 220 to the display device 100 until the electronic pen 200 is connected to the display device 100 via the communication interface 230.

The control unit 250 may compare the user's biometric information acquired by the first sensor 210 with the user's biometric information pre-stored in the memory unit 240, and transmit the comparison result to the display device 100 via the communication interface 230.

In one embodiment, the display device includes a touch display that accepts input from an electronic pen; a receiving unit that receives from the electronic pen first data corresponding to the detection results of biometric information of a user holding the electronic pen and second data corresponding to the detection results of at least one of acceleration, writing pressure, and angular velocity; and a control unit that, if the first data satisfies a predetermined condition, executes a process to link user information to a first image input by the electronic pen based on the second data.

Figure 5 is a diagram showing an example of the operation of a display device according to one embodiment. Also, Figure 6 is a diagram showing, in chronological order, an example of how an image is drawn on the touch display 110 of the display device 100 in steps S101 to S105 of Figure 5.

In this embodiment, the display device 100 and electronic pen 200 are wirelessly connected via short-range wireless communication. The electronic pen 200 has a fingerprint sensor as the first sensor 210 and an acceleration sensor as the second sensor 220. The display device 100 displays an application screen ((a) in Figure 6) that supports handwriting operations on the touch display 110. In (a) in Figure 6, the display device 100 displays a frame 300 on the touch display 110 that indicates an area for accepting electronic signatures. A user can sign by using the electronic pen 200 to handwrite in the area within the frame 300 on the touch display 110.

In this embodiment, an application that supports handwriting operations is, for example, an application that supports electronic signatures, such as an electronic contract application, but is not limited to this as long as it allows input operations using an electronic pen. Furthermore, in this embodiment, a case is described in which characters are drawn on the touch display 110 using the electronic pen 200, but this is not limiting. For example, the display device of the present disclosure can also be applied to a case in which a picture or a work that combines pictures and characters is drawn on the touch display 110 using the electronic pen 200.

In step S101, the control unit 140 receives the first data (fingerprint data) acquired by the first sensor 210 of the electronic pen 200 via the communication interface 120.

In step S102, the control unit 140 compares the received first data (fingerprint data) with the user's fingerprint information pre-stored in the memory unit 130, and determines whether the user holding the electronic pen 200 is a legitimate user, thereby authenticating the user. In other words, the control unit 140 authenticates the user based on the received first data (fingerprint data).

In step S102, if the first data (fingerprint data) matches the user's fingerprint information pre-stored in the memory unit 130 (hereinafter also referred to as a case where user authentication is successful), the control unit 140 proceeds to step S103. On the other hand, if the first data (fingerprint data) does not match the user's fingerprint information pre-stored in the memory unit 130 (hereinafter also referred to as a case where user authentication is unsuccessful), the control unit 140 returns to step S101. If user authentication is unsuccessful, the control unit 140 may display a notification on the touch display 110 prompting the user to authenticate the user using the first sensor 210.

The process of comparing the first data (fingerprint data) acquired by the first sensor 210 with the user's fingerprint information pre-stored in the memory unit 130 may be performed by the electronic pen 200 (control unit 250) instead of the display device 100 (control unit 140). That is, the electronic pen 200 (control unit 250) may perform a process of comparing the first data (fingerprint data) acquired by the first sensor 210 with the user's fingerprint information pre-stored in the memory unit 130, and transmit the comparison result to the display device 100 (control unit 140) via the communication interface 230. In this case, the comparison result is, for example, data indicating whether the first data (fingerprint data) matches the user's fingerprint information pre-stored in the memory unit 130. The control unit 140 may then determine whether user authentication was successful based on the comparison result received from the electronic pen 200 (control unit 250) via the communication interface 120.

In step S103, the control unit 140 starts acquiring the second data and proceeds to step S104. That is, the control unit 140 requests the electronic pen 200 (control unit 250) to transmit the second data via the communication interface 120. In response to this request, the electronic pen 200 (control unit 250) transmits acceleration data, which is data indicating the acceleration acquired by the second sensor 220 (acceleration sensor), to the display device 100 (control unit 140) via the communication interface 230. The electronic pen 200 (control unit 250) periodically transmits the acceleration data to the display device 100 (control unit 140). The control unit 140 periodically receives the acceleration data from the electronic pen 200 (control unit 250).

Also, in step S103, the control unit 140 starts control to draw an object on the display unit 112 based on the position information of the electronic pen or the like received from the touch panel 111. That is, in step S103, when the user uses the electronic pen 200 to perform an operation to draw a specific object on the touch display 110 (hereinafter also referred to as a handwriting operation), the control unit 140 draws the object on the touch display 110 in accordance with the handwriting operation.

Here, Figure 6(a) shows the display state of the touch display 110 immediately after step S103 is executed. Then, after step S103, when the user uses the electronic pen 200 to perform a handwriting operation to sign within the frame 300 in Figure 6(a), the control unit 140 draws an image corresponding to the handwriting operation within the frame 300, as shown in Figure 6(b).

In step S104, the control unit 140 determines whether the electronic pen 200 has stopped being used based on the second data (acceleration data). In other words, the control unit 140 references the second data (acceleration data) over time to estimate a state in which handwriting operation has stopped. Specifically, the control unit 140 determines that the electronic pen 200 is being used if it determines, based on the second data (acceleration data), that the acceleration occurring in the electronic pen 200 is equal to or greater than a predetermined value. The control unit 140 determines that the electronic pen 200 has stopped being used if it determines, based on the second data (acceleration data), that the acceleration occurring in the electronic pen 200 is less than the predetermined value. Here, the control unit 140 may determine that the electronic pen 200 has stopped being used if a predetermined time has elapsed since the acceleration occurring in the electronic pen 200 became less than the predetermined value based on the second data (acceleration data). If the control unit 140 determines that the electronic pen 200 has stopped being used, the process proceeds to step S105. When proceeding to step S105, the control unit 140 may transmit a signal instructing the electronic pen 200 (control unit 250) to stop transmitting the second data (acceleration data) (hereinafter also referred to as a second data transmission stop signal) via the communication interface 120. When the electronic pen 200 (control unit 250) receives the second data transmission stop signal from the display device 100, it may stop transmitting the second data (acceleration data) to the display device 100.

The second sensor 220 may use at least one of a pen pressure sensor and an angular velocity sensor instead of or as a supplement to the acceleration sensor.

Here, the control unit 140 may determine whether the electronic pen 200 is no longer in use as follows.

Whether the electronic pen 200 is no longer in use can be determined, for example, based on the second data (acceleration data). The acceleration indicated by the second data (acceleration data) indicates a unique pattern of time-varying acceleration corresponding to the movement of the electronic pen 200. Specifically, this acceleration periodically fluctuates over time in accordance with the movement of the electronic pen 200. The memory unit 130 of the display device 100 may store, as registered patterns, typical patterns of acceleration corresponding to the state of use of the electronic pen 200 (e.g., a state in which the electronic pen 200 is placed on a desk or the like, a state in which the electronic pen 200 is in free fall, or a state in which the user is holding the electronic pen 200 without using it). The control unit 140 may determine whether the pattern of time-varying acceleration indicated by the second data (acceleration data) (hereinafter also referred to as the detection pattern) indicates a registered pattern of a predetermined state. For example, when the sum of the absolute values of the differences between the detection pattern and the registered pattern at each time point satisfies a predetermined criterion, the control unit 140 determines that the detection pattern indicates the registered pattern. For example, when the detection pattern indicates a registered pattern in which the electronic pen 200 is placed on a desk or the like, the user moves their hand to place the electronic pen 200 on a desk or the like, the electronic pen 200 is falling freely, or the user continues to hold the electronic pen 200 in their hand without using it, the control unit 140 may determine that the electronic pen 200 is no longer in use.

Here, the display device 100 may be equipped with an alarm unit such as a vibrator, speaker, or LED light. The control unit 140 may notify the user via the alarm unit of the display device 100 or the touch display 110 that it has determined that the electronic pen 200 is in use, from the time the control unit 140 starts acquiring the second data (step S103) until it determines that the electronic pen 200 is no longer in use (step S104 YES). The electronic pen 200 may be equipped with an alarm unit such as a vibrator, speaker, or LED light. The electronic pen 200 may notify the user via the alarm unit of the electronic pen 200 that it has determined that the electronic pen 200 is in use, from the time the control unit 140 starts transmitting the second data (step S103) until it determines that the electronic pen 200 is no longer in use (step S104 YES). For example, the electronic pen 200 (control unit 250) may notify the user via the notification unit of the electronic pen 200 that the display device 100 has determined that the electronic pen 200 is being used from the time it receives a signal from the display device 100 requesting transmission of the second data until it receives a signal from the display device 100 to stop transmission of the second data.

In step S105, control unit 140 executes a process of linking a first image, which is an image drawn before determining in step S104 that electronic pen 200 has not been used, with user information. Specifically, control unit 140 stores a management table linking the ID (identifier) of the first image with the user information in storage unit 130 of display device 100. Note that control unit 140 may execute a process of embedding text data indicating the user information in data indicating the first image as the process of linking the first image with the user information. Control unit 140 may execute a process of transmitting a transaction requesting the issuance of an NFT (Non Fungible Token) including the first image and the user information to a blockchain network of an existing NFT issuing company as the process of linking the first image with the user information.

User information includes at least one of the following: user ID, user face image, user name, user affiliation, user biometric information, and user job title.

Here, Figure 6(c) shows the display state of touch display 110 when control unit 140 determines in step S104 that electronic pen 200 is not being used. In other words, control unit 140 performs processing to associate user information with image 301 (see Figure 6(c)) drawn up until it is determined in step S104 that electronic pen 200 is not being used, as the first image. Note that Figure 7 shows an example of only the first image (image 301). Here, control unit 140 may perform processing to associate a part or the entire application screen including image 301 with user information, as the first image, as shown in Figure 8.

As described above, if user authentication is successful, the display device 100 links the user's information to the first image drawn until the second data meets the specified conditions.

With this type of display device 100, once the user has finished writing on the first screen, the user's information can be linked to that first image, which is useful in that it allows the user to link the user's information to the image without having to perform any complicated operations.

In one embodiment, when a first image is input into an application for which a predetermined authority level is set, if the first data satisfies a predetermined condition but the user's authority does not meet the predetermined authority level, the control unit of the display device may display a second image on the touch display indicating that the first image cannot be input, without performing a process to link the first image to the user's information.

The second image may be an image that notifies the user that the first image cannot be input; in other words, an image that notifies the user that the user does not have the authority to input the first image. The second image may be composed of at least one of text, symbols, illustrations, and photographs.

Figure 9 is a diagram showing an example of the operation of a display device according to one embodiment.

Note that in Figure 9, explanations of steps that overlap with the example of operation in Figure 5 will be omitted.

The display device 100 of this embodiment displays, on the touch display 110, the screen of an application that supports the above-mentioned handwriting operation. Furthermore, the application has an authorization level set to permit input via the touch display 110. If the user's authorization satisfies the authorization level, the application permits input to the application screen via the touch display 110.

In step S106, the control unit 140 determines whether the authority of the user authenticated in step S102 satisfies a predetermined authority level. Specifically, the control unit 140 references the user authority information stored in the memory unit 130 and determines whether the authority of the user authenticated in step S102 satisfies the authority level set for the application.

In step S106, if the control unit 140 determines that the user's authority satisfies the authority level, the process proceeds to step S103. On the other hand, in step S106, if the control unit 140 determines that the user's authority does not satisfy the authority level, the process proceeds to step S107.

In step S107, the control unit 140 causes the touch display 110 to display a second image indicating that the first image cannot be input, and then ends the process. For example, the control unit 140 may cause the touch display 110 to display the second image 302, as shown in FIG. 10. Specifically, the control unit 140 causes the touch display 110 to display the second image 302, which notifies the user that the first image cannot be input, by superimposing it on at least a portion of the area for inputting the first image (the area within the frame 300). Note that the transparency of the second image 302 may be set to a value greater than 0, allowing the image located below the second image 302 to be seen.

Furthermore, if the control unit 140 determines in step S102 that the user's authority does not satisfy the predetermined conditions, it may reduce the illumination of the touch display 110 or stop displaying the screen of the application. If the control unit 140 determines in step S102 that the user's authority does not satisfy the predetermined conditions, it may change the display mode, such as by changing the color of the area within the frame 300. In other words, by controlling in this manner, it may be possible to notify the user that they do not have the authority to input the first image.

As described above, the display device 100 does not accept input from a user whose authority does not meet the authority level set for the application. This type of display device 100 can restrict the input of first images by users other than those with the specified authority.

In one embodiment, the display device may display a first image and a third image indicating that the first image is linked to user information on a touch display.

The third image may be composed of, for example, at least one of text, symbols, illustrations, and photographs.

Figure 11 is a diagram showing an example of the operation of a display device according to one embodiment.

In step S108, the control unit 140 determines whether user information is linked to the image displayed on the touch display 110. If user information is linked to the image displayed on the touch display 110 in step S108, the control unit 140 proceeds to step S109. On the other hand, if user information is not linked to the image displayed on the touch display 110 in step S108, the control unit 140 ends the processing. Note that the control unit 140 may perform the processing of step S108, for example, when a triggering user operation is received via the touch display 110, but may also perform the processing at various times.

In step S109, the control unit 140 causes the touch display 110 to display a third image indicating that the image is linked to user information. For example, as shown in FIG. 12, the control unit 140 causes the third image 303 to be displayed near the image linked to the user information (i.e., the first image 301). Note that the control unit 140 may also display the third image 303 superimposed on the first image 301. The third image 303 may also be a frame that surrounds the first image 301.

Furthermore, in step S109, control unit 140 may change the display mode of first image 301. For example, in step S109, control unit 140 may cause first image 301 to blink, or may change the color constituting first image 301.

With this type of display device 100, the user can intuitively understand which of the images displayed on the touch display 110 are linked to the user's information.

In one embodiment, when the touch display detects an input selecting the first image or the third image, the display device may cause the touch display to display a fourth image showing user information.

The fourth image includes information indicating at least one of the user ID, the user's facial image, the user's name, the user's affiliation, the user's biometric information, and the user's job title. The fourth image may be composed of at least one of text, symbols, illustrations, and photographs.

Figure 13 is a diagram showing an example of the operation of a display device according to one embodiment.

Note that in Figure 13, explanations of steps that overlap with the example of operation in Figure 11 will be omitted.

In step S110, if the control unit 140 detects an operation to select the first image 301 via the touch display 110, the process proceeds to step S111. In step S110, if the control unit 140 detects an operation to select the third image 303 via the touch display 110, the process proceeds to step S111. For example, the operation to select the first image 301 or the third image 303 may be an operation to tap the first image 301 or the third image 303.

In step S111, the control unit 140 causes the touch display 110 to display a fourth image 304 showing user information. Specifically, the control unit 140 causes the touch display 110 to display the fourth image 304 near the first image 301 as shown in FIG. 14. The control unit 140 may also cause the touch display 110 to display the fourth image 304 superimposed on at least a portion of the first image 301.

With this type of display device 100, the user can easily obtain information about the user who created the first image 301.

In one embodiment, when the touch display detects an operation to edit the first image, the display device determines whether or not to allow input of an operation to edit the first image based on user information.

Furthermore, when the touch display detects an operation to edit the first image, the display device may display a fifth image on the touch display prompting input of the first data.

The display device also allows editing of the first image if the first data input after displaying the fifth image satisfies predetermined conditions.

Note that the operation of editing the first image includes, for example, at least one of the following: an operation of deleting at least a portion of the first image; an operation of copying at least a portion of the first image; an operation of changing the display mode of at least a portion of the first image (e.g., editing to change the color, line thickness, etc.); an operation of downloading the first image; and an operation of adding an image to the first image by handwriting.

The fifth image may be any image that prompts the user to input the first data, in other words, prompts the user to input biometric information, and may be composed of, for example, at least one of letters, symbols, illustrations, and photographs.

Figure 15 is a diagram showing an example of the operation of a display device according to one embodiment. The display device 100 displays the screen shown in Figure 16 on the touch display 110.

In step S112, the control unit 140 detects an operation to edit the first image 301. Specifically, the control unit 140 detects an operation in which the user presses and holds the first image 301 on the touch display 110 to delete part of the first image 301, and the process proceeds to step S113.

In step S113, the control unit 140 displays a fifth image 305 on the touch display 110, prompting the user to input the first data, as shown in FIG. 16, and then proceeds to step S114.

In step S114, the user is prompted by the fifth image 305 to input fingerprint information into the display device 100 using the first sensor 210 of the electronic pen 200. That is, the control unit 140 acquires the first data (fingerprint data) from the electronic pen 200 via the communication interface 120, and proceeds to step S115. Note that the display device 100 may be equipped with a biometric sensor such as a fingerprint sensor. In step S114, the display device 100 may acquire the first data from a biometric sensor equipped in the device itself.

In step S115, the control unit 140 determines whether the first data (fingerprint data) input in step S112 matches the user's fingerprint information linked to the first image 301. If the input first data (fingerprint data) matches the user's fingerprint information linked to the first image 301 in step S115, the process proceeds to step S116. On the other hand, if the input first data (fingerprint data) does not match the user's fingerprint information linked to the first image 301 in step S115, the control unit 140 returns to step S113. Note that if the input first data (fingerprint data) does not match the user's fingerprint information linked to the first image 301, the control unit 140 may execute a process to notify a notification destination (e.g., email address, etc.) previously registered by the user linked to the first image 301 that someone else is attempting to edit the first image. The control unit 140 may count the number of times that the input first data (fingerprint data) does not match the user's fingerprint information linked to the first image 301. When the counted number reaches a predetermined number, the control unit 140 may end the process without returning to S113, and thereafter may prevent the first image 301 from being edited for a predetermined period of time.

In step S116, the control unit 140 allows the user to edit the first image 301. Specifically, the control unit 140 deletes the first image 301. Note that in step S116, the control unit 140 may make the first image 301 editable and delete the first image 301 in response to a user operation received after step S116.

In step S116, the control unit 140 may cause the touch display 110 to display an image indicating that editing of the first image 301 is permitted. Also, in step S116, the display mode of the first image 301 may be changed. For example, the control unit 140 changes the display mode of the first image 301 by displaying the first image 301 in a manner that causes it to periodically vibrate. For example, the control unit 140 changes the display mode of the first image 301 by changing the color of the first image 301.

As described above, the display device 100 can prevent the first image from being altered, deleted, or used by third parties other than the author of the first image.

In this disclosure, descriptions such as "first," "second," "third," "fourth," and "fifth" are identifiers used to distinguish the components. In this disclosure, components distinguished by descriptions such as "first," "second," "third," "fourth," and "fifth" may have their numbers swapped. For example, the first image and the second image may swap their identifiers, "first" and "second." The identifier swapping occurs simultaneously. The components remain distinct even after the identifier swapping. Identifiers may be deleted. Components from which identifiers have been deleted are distinguished by their reference symbols. The identifiers "first," "second," "third," "fourth," and "fifth" used in this disclosure alone should not be used to interpret the order of the components or to justify the existence of identifiers with lower numbers.

A program may be provided that causes a computer to execute each process performed by the display device 100. The program may be recorded on a computer-readable medium. Using the computer-readable medium, the program can be installed on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, but may be, for example, a CD-ROM, DVD-ROM, or other recording medium.

One embodiment has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made without departing from the spirit of the invention.

100: Display device 101: Housing 110: Touch display 111: Touch panel 112: Display unit 120: Communication interface 130: Memory unit 131: Program 140: Control unit 200: Electronic pen 201: Housing 202: Core body 210: First sensor 220: Second sensor 230: Communication interface 240: Memory unit 250: Control unit 300: Frame 301: First image 302: Second image 303: Third image 304: Fourth image 305: Fifth image

Claims (8)

a touch display that accepts input from an electronic pen;
a receiving unit that receives, from the electronic pen, first data corresponding to a detection result of biometric information of a user holding the electronic pen and second data corresponding to a detection result of at least one of acceleration, writing pressure, and angular velocity;
a control unit that executes a process of linking information about the user to a first image input by the electronic pen based on the second data when the first data satisfies a predetermined condition ,
The control unit causes the touch display to display the first image and a third image indicating that the first image is linked to user information.
Display device. The control unit
When the first image is input into an application having a predetermined authority level,
If the first data satisfies a predetermined condition but the user's authority does not meet the predetermined authority level,
The display device according to claim 1 , wherein a second image indicating that the first image cannot be input is displayed on the touch display without performing a process of linking the first image to the user's information. The display device according to claim 1 , wherein the control unit causes the touch display to display a fourth image showing information about the user when the touch display detects an input selecting the first image or the third image. The display device according to claim 1, wherein the control unit, when the touch display detects an operation to edit the first image, causes the touch display to display a fifth image prompting input of the first data. The display device according to claim 4 , wherein the control unit permits editing of the first image when the first data input after the fifth image is displayed satisfies a predetermined condition. An electronic pen and
a display device having a touch display that accepts input from the electronic pen;
Equipped with
The electronic pen is
a first sensor that acquires biometric information of a user holding the electronic pen;
at least one second sensor that detects at least one of acceleration, writing pressure, and angular velocity;
a transmitter that transmits, to the display device, first data corresponding to a detection result of the first sensor and second data according to a detection result of the at least one second sensor;
and
The display device includes:
a receiving unit that receives the first data and the second data;
A control unit;
and
The control unit
If the first data satisfies a predetermined condition,
executes a process of linking information about the user to a first image input by the electronic pen based on the second data;
The control unit causes the touch display to display the first image and a third image indicating that the first image is linked to information of the user.
Pen input system. An information processing method executed by a control unit of a display device including a receiving unit and a touch display,
When the first data corresponding to the biometric information of the user acquired by the electronic pen, which is received by the receiving unit from the electronic pen, satisfies a predetermined condition,
a process of associating information about the user with a first image input to the touch display by the electronic pen based on second data corresponding to at least one of detection results of acceleration, writing pressure, and angular velocity applied to the electronic pen, the second data being received by the receiving unit from the electronic pen ;
and displaying, on the touch display, the first image and a third image indicating that the first image is linked to information of the user.
Information processing methods. When the first data corresponding to the biometric information of the user acquired by the electronic pen, which is received by the receiving unit from the electronic pen, satisfies a predetermined condition,
executes a process of linking information about the user to a first image input to a touch display by the electronic pen, based on second data corresponding to at least one of detection results of acceleration, writing pressure, and angular velocity applied to the electronic pen, which data is received by the receiving unit from the electronic pen;
The first image and a third image indicating that the first image is linked to information of the user are displayed on the touch display.
program.