JP6897976B2 - Device failure detector, failure detection method, and failure detection program - Google Patents

Description

The present invention relates to a device failure detection device, a failure detection method, and a failure detection program.

As is well known, there are many processing devices in the world that need to be operated by a user. For example, such processing devices include cash processing devices such as ATMs (automatic teller's machines), store terminals such as multimedia terminals and POS (point of sales) terminals, and the like.

Such a processing device that the user needs to operate includes a plurality of input devices (operation units) for the user to operate. Examples of the plurality of input devices (operation units) include a touch panel, a numeric keypad, and a card reader for reading card information. The touch panel has a plurality of buttons (software keys). The numeric keypad consists of multiple hardware keys. The card reader has an insertion slot for inserting a card such as a cash card.

In the present specification, the input device (operation unit) is also referred to as a "device". Needless to say, the device is not limited to the above, and may include an arbitrary input device (operation unit) for the user to operate.

In this type of processing device, failure detection of each device is generally performed by a command / response method. However, as will be described in detail later with reference to the drawings, this command / response method cannot detect the following two types of failure phases.

The first failure phase is a failure phase in which the specific device does not operate even though the user operates the specific device while operating the processing device. The second failure phase is a failure phase in which a device operates without permission even though the user is not operating any device. As described above, the command / response method cannot detect the first and second failure phases. Therefore, in such a case, the failure of the device of the processing device can be detected only by the declaration of the operator (user) who operated the processing device. Therefore, in the related technology, there is a problem that the failure cannot be detected on the operation side in advance and the maintenance response may be delayed.

In addition, various prior art documents related to the present invention are known.

For example, Patent Document 1 discloses a mobile terminal including an operation unit, a display unit, and a control unit. The operation unit is an operation interface for the user to operate the mobile terminal. The operation unit is composed of input devices such as a numeric keypad and a touch panel, receives various operations from the user, and outputs signals corresponding to those operations to the control unit. By configuring the display unit with a touch panel display, the display unit and the operation unit may be integrated.

Patent Document 2 discloses a device management system including a network camera that functions as a surveillance camera. In Patent Document 2, a network camera that functions as an imaging means captures a user who operates a network device. When a user operates an MFP (Multifunction Peripheral) as a network device to be managed, an image of an operation record is recorded by a network device that functions as an imaging means.

Further, Patent Document 3 discloses a coordinate input device that reduces malfunction of the information processing device due to erroneous input. The coordinate input device disclosed in Patent Document 3 includes a touch panel, an input coordinate detection unit, a vibration sensor, a waveform pattern storage unit, a vibration waveform identification unit, and a determination unit. The touch panel accepts touch input operations from the operator. The input coordinate detection unit detects the coordinate position based on the electric signal generated when the touch input operation is performed on the touch panel, and outputs the touch input notification signal. The vibration sensor is provided on the surface of the touch panel and detects the vibration transmitted by using the touch panel as a vibration transmission plate when the touch panel is operated. The waveform pattern storage unit stores a waveform pattern obtained by frequency analysis of the operation vibration generated when the touch input operation is performed satisfactorily as a reference waveform pattern. The vibration waveform identification unit frequency-analyzes the vibration detected by the vibration sensor, acquires the detected waveform pattern, compares the detected waveform pattern with the reference waveform pattern, and confirms whether each touch input operation is a normal operation intended by the operator. It identifies whether or not, and outputs the identification result as an operation status notification signal. The determination unit determines whether the touch input operation is valid or invalid based on the touch input notification signal and the operation status notification signal, and outputs the determination result.

Patent Document 4 discloses a "processing device" that improves the availability of an information processing system. The information processing system is made redundant by a plurality of processing devices, and is constructed so that business can be continued even if one of the processing devices fails. The processing device includes a device for constructing the processing device and a processing unit. The device in Patent Document 4 is a functional block such as a front end (FE), an FE relay device, a control unit, a memory, a state management unit, a back end (BE), and a BE relay device. The processing unit detects a device or an abnormality between devices. The processing unit measures the number of detected device abnormalities. When the number of abnormalities of the measured device is equal to or greater than the threshold value, the processing unit logically disconnects the device from the information processing apparatus system. If the number of detected device abnormalities is less than the threshold value, the processing unit reboots the device.

In the processing device of Patent Document 4, the state management unit transmits a state confirmation packet (command) to each functional block by polling. When each functional block receives a status monitoring command, each functional block transmits its own status information (whether or not an error has occurred in its own functional block) to the status management unit. Each functional block notifies the state management unit of information indicating its own abnormal state. When a fatal error occurs, each functional block sends an error notification packet (command). Further, in Patent Document 4, the state confirmation unit transmits a state confirmation packet to the functional blocks by polling to confirm the state of each functional block.

Patent Document 5 discloses a "simple authentication system for an image processing device" that can simplify the complicated password input operation while ensuring security. In Patent Document 5, when the user logs in to the image processing device, the registered password of the user is displayed in the order of the registered gesture patterns on the password input screen displayed on the display means. .. As a result, the user can enter the password on the password entry screen simply by pressing the software key displayed according to the gesture pattern.

JP 2015-121969 JP-A-2009-193169 Japanese Unexamined Patent Publication No. 2009-026174 Japanese Unexamined Patent Publication No. 2014-191401 Japanese Unexamined Patent Publication No. 2014-075011

However, each of Patent Documents 1 to 5 has the following problems.

Patent Document 1 merely discloses that a signal corresponding to an operation in the operation unit is output to the control unit. In other words, Patent Document 1 does not recognize, disclose, or suggest any technical idea for detecting a device failure, which the present invention is trying to solve.

Patent Document 2 merely discloses a network camera (surveillance camera) that captures a user who operates a network device. Patent Document 2 does not recognize, disclose, or suggest any technical idea for detecting a device failure, which the present invention is trying to solve.

Patent Document 3 merely discloses a technique for identifying whether or not a touch input operation is a normal operation by duplicating the position detection by a touch panel and a vibration sensor. Therefore, Patent Document 3 merely discloses that the erroneous input is identified and excluded, and cannot detect the above-mentioned two types of failure phases that the present invention intends to detect.

Patent Document 4 merely discloses a technique of detecting an abnormality occurring in a device inside a processing device by the above-mentioned command / response method and detecting the abnormality detection of the device by the number of times. Therefore, the processing apparatus of Patent Document 4 cannot detect the above-mentioned two types of failure phases that the present invention intends to detect.

Further, the device of Patent Document 4 is intended for a device capable of detecting a failure (failure) on the side (internal) of the processing device. On the other hand, the device targeted by the present invention is a device (input device) in which a failure cannot be detected on the side (internal) of the processing device. In other words, the device of Patent Document 4 does not include the input device (operation unit) which is the object of the present invention, and is different from the device which is the object of the present invention.

Patent Document 5 merely discloses a technical idea using a gesture pattern as an input of a password required for authentication. Patent Document 5 does not recognize, disclose, or suggest any technical idea for detecting a device failure, which the present invention is trying to solve.

An object of the present invention is to provide a failure detection device, a failure detection method, and a failure detection program for a device that solves the above-mentioned problems.

As one aspect of the present invention, the device failure detection device is a failure detection device that detects a failure of a plurality of devices that can be operated by the user in the processing device, and the operation status of the plurality of devices by the user. An operation gesture in which the operation status of the user operating the plurality of devices is represented by a camera image taken by the camera in advance before actually operating the processing device. A user operation pattern storage unit that stores a plurality of user operation patterns obtained by patterning for each of a plurality of devices in advance is included, and the user operation is performed based on a camera image taken by the camera during operation of the processing device. By referring to the plurality of user operation patterns stored in the pattern storage unit, an operation device identification unit that identifies which device the user is operating and outputs a user operation detection result signal; With the device output detection unit that detects the signal output from any of the devices and outputs the device output detection signal; the device output detection signal is compared with the user operation detection result signal, and any of the plurality of devices. It is provided with a failure determination control unit for determining the presence or absence of such a failure.

As another aspect of the present invention, the device failure detection method is a failure detection method for detecting a failure of a plurality of devices that can be operated by a user in the processing device, and is performed before the processing device is actually operated. the a plurality of camera capable of photographing an operation status by the user to the device, an operation gesture represented by a camera image shooting Me beforehand the operation status of the user operation on the plurality of devices, the A plurality of user operation patterns obtained by patterning for each of a plurality of devices are stored in the user operation pattern storage unit in advance; the user is based on a camera image taken by the camera during the operation of the processing device. By referring to the plurality of user operation patterns stored in the operation pattern storage unit, it is possible to identify which device the user is operating and output a user operation detection result signal; any one of the plurality of devices. The signal output from the device is detected, and the device output detection signal is output; the device output detection signal is compared with the user operation detection result signal, and the presence or absence of failure of any of the plurality of devices is determined.

As another aspect of the present invention, the fault detection program is a fault detection program that causes a computer to detect a fault of a plurality of devices that can be operated by a user in a processing device, and is before actually operating the processing device. , in the plurality of camera capable of photographing an operation status by the user to the device, an operation gesture represented by a camera image shooting Me beforehand the operation status of the user operation on the plurality of devices, A user operation pattern storage procedure in which a plurality of user operation patterns obtained by patterning each of the plurality of devices are stored in the user operation pattern storage unit in advance; a camera photographed by the camera during the operation of the processing device. An operation of identifying which device the user is operating and outputting a user operation detection result signal by referring to the plurality of user operation patterns stored in the user operation pattern storage unit based on the image. The device identification procedure; the device output detection procedure that detects the signal output from any of the plurality of devices and outputs the device output detection signal; and the device output detection signal and the user operation detection result signal are compared. Then, the computer is made to execute the failure determination control procedure for determining the presence or absence of failure of any of the plurality of devices.

Further, it is realized by a computer-readable recording medium for recording the program.

According to the present invention, it is possible to detect two types of failure phases that cannot be detected by the command / response method.

It is a flowchart which shows the failure detection flow of the device of the related technology. It is a figure which shows the hardware composition of the processing apparatus to which the failure detection apparatus of the device which concerns on 1st Embodiment of this invention is applied, (A) is the front view of the processing apparatus, (B) is the processing apparatus. It is a panel side view of. It is a block diagram which shows the structure of the failure detection apparatus of the device which concerns on 1st Embodiment of this invention provided in the processing apparatus shown in FIG. It is a block diagram which shows the structure of the failure detection apparatus of the device which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the detailed internal structure of the operation device identification part used for the fault detection apparatus shown in FIG. It is a figure which shows an example of a plurality of user operation patterns stored in the user operation pattern storage part of FIG. FIG. 3 is a block diagram showing a detailed internal configuration of a failure determination control unit used in the failure detection device shown in FIG. It is a flowchart which shows the operation flow of the trouble detection apparatus shown in FIG. It is a block diagram which shows an example of the hardware of the information processing apparatus (computer) when the failure detection apparatus shown in FIG. 3 or FIG. 4 is realized by software.

[Related technology]
In order to facilitate the understanding of the present invention, first, the related art of the present invention will be described.

FIG. 1 is a flowchart showing a failure detection flow of a device of a related technology. As shown in FIG. 1, in the device failure detection method of the related technology, a user operation command is issued with a user operation (step S101) as a trigger (step S102). The device operation response is detected in response to the issuance of the user operation command (step S102) (step S103). If a response is detected (YES in step S103), the processing device continues normal operation (step S104). On the other hand, if the response cannot be detected (NO in step S103), the processing device notifies an error (step S105).

In the device failure detection method of the related technology shown in FIG. 1, the first and second failure phases as described below cannot be detected by the processing device.

In the first failure phase, the operation command of step S102 is not issued from the specific device and the specific device does not operate even though the user operates the specific device during the operation of the processing device (step S101). It is a failure phase. The second failure phase is a failure phase in which an operation command is issued from a device having an operation command in step S102 and the device operates even though the user has not operated the device in step S101.

As described above, in the device failure detection method of the related technology, the first or second failure phase cannot be detected by the processing device. As a result, there are cases where a device failure can be detected only by the declaration of the operator who operated the processing device.

[Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
The failure detection device of the device according to the first embodiment of the present invention will be described with reference to FIGS. 2 and 3.

FIG. 2 is a diagram showing a hardware configuration of a processing device 200 to which the failure detection device of the device according to the first embodiment is applied. In FIG. 2, (A) is a front view of the processing device 200, and (B) is a panel side view of the processing device 200. FIG. 3 is a block diagram showing a configuration of the fault detection device 300 according to the first embodiment.

As shown in FIG. 2A, the illustrated processing device 200 may consist of, for example, a cash processing device installed in a convenience store.

The processing device 200 mounts the camera 202. The processing device 200 includes visors 204 on both side surfaces thereof. The processing device 200 includes a touch panel display 206 that can be operated by the user between the visors 204. Further, the processing device 200 includes a numeric keypad 208 at the lower part of the touch panel display 206. Although not shown in FIG. 2A, the processing device 200 includes a card reader 210 inside the processing device 200, which can read information on a card such as a cash card. The processing device 200 is provided with an insertion port 210a for the card reader 210 in front of the processing device 200.

As shown in FIG. 2B, the touch panel display 206 includes a touch panel 212, a display panel 214, and a viewing angle limiting filter 216. A camera 202 is mounted on the upper part of the touch panel display 206. The camera 202 can capture a shooting range as shown in FIG. 2 (A). That is, the camera 202 can shoot a shooting range including the touch panel display 206, the numeric keypad 208, and the insertion slot 210 of the card reader.

Therefore, the illustrated processing device 200 includes a touch panel 210, a numeric keypad 208, and a card reader 210 (insertion port 210a) as devices.

The device failure detection method according to the first embodiment of the present invention uses the camera 202 mounted on the processing device 200 to detect the operation operation of the device by the user. Then, the device failure detection method detects an output signal (operation response) from the device when the user's operation operation is detected. In the device failure detection method, the failure detection device 300 shown in FIG. 3 performs device failure detection based on the coincidence or disagreement between the operation operation detection and the output signal detection (operation response detection).

As described above, in the related technology, the first failure phase in which the device does not operate even though the user operates the processing device and the second in which the device operates even though the user does not operate the processing device. At present, the failure phase can only be detected by the operator's declaration.

On the other hand, in the first embodiment, the failure detection device 300 shown in FIG. 3, which will be described later, autonomously detects the failure of the device by matching or disagreeing between the operation motion detection and the device motion response detection. Realizes HW (hardware) reset and failure detection.

FIG. 3 is a block diagram showing a device failure detection device 300 provided in the processing device 200 of FIG. 2 (A).

The device failure detection device 300 includes an operation device identification unit 302, a device output detection unit 304, and a failure determination control unit 306.

The operation device identification unit 302 receives a camera image of the operation status of the user's device captured by the camera 202, identifies which device the user is operating based on the camera image, and detects the user operation. Output the result signal. This user operation detection result signal represents the operation operation detection, that is, the user operation.

The device output detection unit 304 detects a signal (operation response) output from any of the plurality of devices 208, 210, and 212, and outputs a device output detection signal. This device output detection signal represents the operation response detection of the device, that is, the device state.

The failure determination control unit 306 compares the device output detection signal with the user operation detection result signal, and determines whether or not any of the plurality of devices 208, 210, and 212 has a failure. That is, when the device output detection signal and the user operation detection result signal do not match, the device is in the first failure phase or the second failure phase.

Specifically, when the user operation detection result signal is received but the device output detection signal is not received, the failure determination control unit 306 can determine that the device is in the first failure phase. On the other hand, when the user operation detection result signal is not received but the device output detection signal is received, the failure determination control unit 306 can determine that the device is in the second failure phase.

As described above, according to the first embodiment, the processing device 200 can determine the failure in the first and second failure phases without the user's declaration.

[Second Embodiment]
The fault detection device 300A of the device according to the second embodiment of the present invention will be described with reference to FIGS. 4 to 8.

FIG. 4 is a block diagram showing a configuration of a failure detection device 300A for the device according to the second embodiment. FIG. 5 is a block diagram showing in detail the internal configuration of the operation device identification unit 302 used in the failure detection device 300A shown in FIG. FIG. 6 is a diagram showing an example of a plurality of user operation patterns stored in the user operation pattern storage unit (described later) of the operation device identification unit 302. FIG. 7 is a block diagram showing in detail the internal configuration of the failure determination control unit 306 used in the failure detection device 300A shown in FIG. FIG. 8 is a flowchart showing an operation flow of the failure detection device 300A shown in FIG.

The illustrated fault detection device 300A is different from the fault detection device 300 shown in FIG. 3 except that it further includes a CPU (central processing unit) 301, a device reset control unit 308, and an error notification unit 310. It has the same configuration as the fault detection device 300 shown in FIG. 3 and operates. Hereinafter, in order to simplify the description, only the points different from the failure detection device 300 shown in FIG. 3 will be described in detail.

The CPU 301 mainly controls the entire failure detection device 300A.

The device reset control unit 308 resets the device when the detected user operation state and the device state do not match. In other words, the device reset control unit 308 resets the corresponding device in response to the reset signal supplied from the failure determination control unit 306, as will be described later.

The error notification unit 310 notifies when it is determined that there is a device failure. More specifically, the failure determination control unit 306 first counts the number of failure result determination signals indicating the determination result of the presence or absence of a failure. Then, the failure determination control unit 306 issues an error message to the error notification unit 310 when the counted number of times reaches a predetermined number of times.

The fault detection device 300A solves the problems of the related technology shown in FIG. 1 as follows. That is, as will be described later, the operation pattern of the target device is saved in advance in the user operation pattern storage unit (described later) of the operation device identification unit 302. Then, the operation device identification unit 302 compares the user operation shown in the camera image captured by the camera 202 mounted on the processing device 200 shown in FIG. 2 with the operation pattern, and determines whether the user is operating the device. Judge whether or not. From this, the problem of the above-mentioned related technology is solved.

FIG. 5 is a block diagram showing in detail the internal configuration of the operation device specifying unit 302 shown in FIG.

The operation device identification unit 302 includes a camera control unit 502, a user operation detection unit 504, and a user operation pattern storage unit 506.

The camera control unit 502 controls the camera 202 mounted on the processing device 200 and acquires a camera image.

The user operation pattern storage unit 506 displays an operation gesture in which an operation status in which a user operates on a plurality of devices 212, 210, 208 is represented by a camera image previously captured by the camera 202, and a plurality of devices 212, 210, 210. A plurality of user operation patterns obtained by patterning each 208 are stored.

The user operation detection unit 504 detects the user operation from the acquired camera image. More specifically, the user operation detection unit 504 identifies a device operated by the user by referring to a plurality of user operation patterns stored in the user operation pattern storage unit 506 from the acquired camera image, and the user operates the user. Outputs the operation detection result signal.

In the case of this example, the operation gesture includes the first to third gestures described below. The first gesture represents a first operation situation in which the user operates the touch panel 212. The second gesture represents a second operation situation in which the user inserts the card into the insertion slot 210a of the card reader 210. The third gesture represents a third operation situation in which the user operates the numeric keypad 208.

FIG. 6 is a diagram showing an example of a plurality of user operation patterns stored in the user operation pattern storage unit 506.

As shown in FIG. 6, in the user operation pattern storage unit 506, a plurality of operation types and a plurality of user operation patterns are registered in a one-to-one association. For example, when the operation type is "touch panel operation", the first gesture is registered as the corresponding user operation pattern. Similarly, when the operation type is "card reader operation", the second gesture is registered as the corresponding user operation pattern. When the operation type is "numeric keypad operation", a third gesture is registered as the corresponding user operation pattern.

More specifically, the first gesture is a pattern of the movement of the user trying to press the surface of the touch panel 212, and is a pattern of the movement when the user tries to press the surface of the panel. It is a thing. The second gesture is a pattern of the user's movement to insert the card into the insertion slot 210a of the card reader 210. The third gesture is a pattern of the user's movement to press the numeric keypad 208.

Each user operation pattern is data representing the characteristics of the operation status of the user operation corresponding to each operation type. This user operation pattern is data obtained by previously analyzing the camera image captured by the camera 202 using a software image analysis system and recognizing what kind of operation gesture it was.

FIG. 7 is a block diagram showing in detail the internal configuration of the failure determination control unit 306 shown in FIG.

The failure determination control unit 306 includes a device operation determination unit 402, an operation device determination unit 404, a failure determination unit 406, and a recovery control unit 408.

The device operation determination unit 402 detects the signal output from the device and determines whether or not the device is operating. More specifically, the device operation determination unit 402 detects the device output detection signal supplied from the device output detection unit 304 and determines whether or not any of the plurality of devices 212, 210, and 208 is operating. Then, the device operation determination result signal indicating the device operation state is output.

The operation device determination unit 404 determines the device being operated from the user operation detected by the user operation detection unit 504 of the operation device identification unit 302. More specifically, the operation device determination unit 404 determines the device operated by the user in response to the user operation detection result signal sent from the user operation detection unit 504, and indicates the user operation state. Outputs the status judgment result signal.

The failure determination unit 406 compares the device operating state with the user operating state to determine the operating state of the device. More specifically, the failure determination unit 406 determines a match or disagreement between the user operation state and the device operation state by comparing the device operation determination result signal with the user operation state determination result signal, and determines a plurality of matches. A failure determination result signal indicating the determination result of the presence or absence of failure of any of the devices 212, 210, and 208 is output.

The recovery control unit 408 resets the device when the detected user operation state and the device operation state do not match. More specifically, the recovery control unit 408 outputs the above reset signal for recovering the device determined to be a failure when the failure determination unit 406 determines that the user operation state and the device operation state do not match. To do.

The outline of the operation of the second embodiment will be described.

First, it is assumed that the user operation detection unit 504 of the operation device identification unit 302 detects the user operation from the camera image captured by the camera 202 mounted on the processing device 200 shown in FIG. At this time, the operation device determination unit 404 determines the operation target device in response to the user operation detection result signal. The device output detection unit 304 monitors the output signal from the determined operation target device, and the device operation determination unit 402 determines the operation state of the device. The failure determination unit 406 determines whether or not there is a device failure based on the detected user operation state and device operation state of the operation target device. As a result, the processing device 200 can determine the failure without reporting the user.

The second embodiment of the present invention is characterized by the following.

As described above, the user operation pattern storage unit 506 of the operation device identification unit 302 stores the operation gestures for operating the device in a pattern in advance. When the user operation detection unit 504 of the operation device identification unit 302 detects a user operation from the camera image of the camera 202 mounted on the processing device 200 shown in FIG. 2, the operation device determination unit 404 operates from a matching pattern. Determine the device to be used. The device output detection unit 304 monitors the output signal from the determined device. The device operation determination unit 402 determines the operation state of the device. The failure determination unit 406 determines whether the user operation state and the device operation state match or do not match. As a result, the device failure can be detected by the processing device 200.

Next, the operation of the second embodiment will be described with reference to the operation flow diagram of FIG.

Prior to the start, a patterned image to be pressed is registered in the user operation pattern storage unit 506 of the failure detection device 300A in advance. In the registration method, device operations are performed from various directions in order to acquire a basic operation pattern, and the pattern when the device operates is stored in the user operation pattern storage unit 506 (step S201).

The user operation detection unit 504 detects the presence or absence of device operation from the camera image captured by the camera 202 (step S202). When the device operation is detected (YES in step S202), the device output detection unit 304 acquires the device output signal from the specified device (step S203). When the device output signal is detected (YES in step S203), the failure determination control unit 306 determines that the operation is normal, and the processing device 200 continues the operation (step S204). When there is no detection of the device output signal (NO in step S203), the failure determination control unit 306 executes the failure determination mode.

In the failure determination mode, the device reset control unit 308 resets the corresponding device in response to the reset signal from the recovery control unit 408 (step S209). Then, the failure determination control unit 306 again indicates the device operation message to the user and prompts the user to perform the device operation again (step S210). If the failure determination unit 406 does not recover after a plurality of times, it determines that the failure has occurred (step S211) and notifies the error notification unit 310 of the error (step S212).

On the other hand, when there is no detection of device operation (NO in step S202), the device output detection unit 304 monitors the device output signal from each device (step S205). When there is no device output signal (NO in step S105), the failure determination unit 406 determines that the user operation state and the device operation state match, and the processing device 200 continues the normal operation (S206). If there is a device output signal (YES in step S205), the user operation state and the device operation state do not match, so the failure determination mode is entered.

In this failure determination mode, the recovery control unit 408 supplies a reset signal to the device reset control unit 308 in order to reset the device from which the device signal was output (step S207). If the failure determination unit 406 does not recover a plurality of times, the failure determination unit 406 determines that the failure has occurred (step S208), and notifies the error notification unit 310 of the error (step S212).

Next, the effect of the second embodiment will be described.

In the related technology, there is no means for detecting the operation status of the user in the device in which the user operation triggers the operation. Therefore, the processing device may not be able to detect a failure in the following two patterns.

1) When the operation detection signal (operation output signal) is not output from the device even though the user is actually operating it.
2) When an operation output signal is output from the device and a malfunction occurs even though the user is not actually operating it.

Since the processing device cannot detect the operation status of the above two patterns in the related technology, it may not be possible to detect the failure of the device. On the other hand, in the second embodiment of the present invention, the device failure can be detected on the processing device side.

As a result, the following effects are expected in the second embodiment.

-It increases the possibility that the device can be recovered by autonomously resetting HW (hardware).
-In the related technology, the failure cannot be found unless the user declares the operation result by himself / herself, but in the second embodiment, the failure can be determined on the processing device side, which facilitates remote maintenance.

Next, a first embodiment of the present invention when applied to the second embodiment will be described. The first embodiment is an example when the device is a touch panel 212.

In this case, in the first embodiment, the operation gesture (first gesture) in which the user operates the touch panel 212 is patterned in advance by the operation device identification unit 302 of FIG. 4 and stored in the user operation pattern storage unit 506. .. In this situation, it is assumed that the user operation detection unit 504 of the operation device identification unit 302 detects the user operation of pressing the touch panel 212 with the camera 202 mounted on the processing device 200 shown in FIG. In this case, the operation device determination unit 404 of FIG. 7 determines that the operation target device is the touch panel 212. The device output detection unit 304 monitors the output signal from the determined touch panel 212, and the device operation determination unit 402 determines the device operation state of the touch panel 212. The failure determination unit 406 determines whether or not the user operation state of the touch panel 212 by the user and the device operation state of the touch panel 212 match or do not match. From this, in the first embodiment, the failure of the touch panel 212 can be detected by the processing device 200.

Hereinafter, the operation of the first embodiment will be described in detail.

Gestures (first gestures) in which the user operates the touch panel 212 are patterned in advance, and are collected as a database in the user operation pattern storage unit 506 of the operation device identification unit 302 of FIG. Here, the gesture to be operated (first gesture) is, as described above, a pattern of the movement of the user trying to press the surface of the touch panel 212, and the user will press the surface of the touch panel 212. It is a pattern of the operation when.

In this way, as shown in FIG. 6, an operation pattern (first gesture) in which the user operates the touch panel 212 is stored in advance in the user operation pattern storage unit 506 of the operation device identification unit 302. The failure determination control unit 306 is detected by the device output detection unit 304 and the user operation state detected by the user operation detection unit 504 based on the camera image captured by the camera 202 mounted on the processing device 200 shown in FIG. It is determined whether or not the user is operating the touch panel 212 by comparing with the operating state of the device. When an error is detected, the failure determination control unit 306 issues an error notification to the error notification unit 310.

As described above, the failure determination control unit 306 includes a device operation determination unit 402, an operation device determination unit 404, a failure determination unit 406, and a recovery control unit 408.

In the case of the first embodiment, the device operation determination unit 402 detects the device output detection signal supplied from the device output detection unit 304, which indicates that the output signal has been output from the touch panel 212, and the touch panel 212 operates. Determine if it is. The operation device determination unit 404 determines that the device being operated is the touch panel 212 from the user operation detection result signal detected by the user operation detection unit 504. The failure determination unit 406 compares the touch panel operating state with the user operating state to determine the operating state of the touch panel 212. The recovery control unit 408 outputs a reset signal for resetting the touch panel 212 when it is determined that the user operation state of the touch panel 212 detected by the failure determination unit 406 and the touch panel operating state do not match.

It is assumed that the camera 202 mounted on the processing device 200 shown in FIG. 2 acquires a camera image showing an operation status in which the user is operating the touch panel 212. At this time, the user operation detection unit 504 refers to the user operation pattern stored in the user operation pattern storage unit 506, and outputs a user operation detection result signal indicating that the user is operating the touch panel 212. In response to this user operation detection result signal, the operation device determination unit 404 determines that the operation target device is the touch panel 212. The device output detection unit 304 monitors the output signal from the determined touch panel 212, and the device operation determination unit 402 determines the operation state of the touch panel 212. The failure determination unit 406 determines the operating state of the touch panel 212 from the detected user operation state of the touch panel 212 and the device operating state of the touch panel 212. From this, the failure determination can be performed by the processing device 200 without the user's declaration. When the failure determination unit 406 determines that the user operation state of the touch panel 212 and the device operation state of the touch panel 212 match, the processing device 200 continues the normal operation. On the other hand, when the failure determination unit 406 determines that there is a mismatch, the recovery control unit 408 performs device recovery control. If they do not match a plurality of times, the failure determination unit 406 determines that an error occurs and issues an error message to the error notification unit 310.

Next, a second embodiment of the present invention when applied to the second embodiment will be described. The second embodiment is an example when the device is a card reader 210.

In this case, in the second embodiment, the gesture (second gesture) in which the user inserts the cash card into the insertion slot 210a of the card reader 210 is patterned in advance by the operation device identification unit 302 of FIG. 4 to store the user operation pattern. It is stored in the unit 506. In this situation, it is assumed that the user operation detection unit 504 of the operation device identification unit 302 detects a user operation of inserting a card into the insertion slot 210a of the card reader 210 by the camera 202 mounted on the processing device 200 shown in FIG. .. In this case, the operation device determination unit 404 of FIG. 7 determines that the operation target device is the card reader 310. The device output detection unit 304 monitors the output signal from the determined card reader 310, and the device operation determination unit 402 determines the operation state of the card reader 310. The failure determination unit 406 determines whether the user operating state of the card reader 310 and the device operating state of the card reader 310 match or do not match. As a result, in the second embodiment, the failure of the card reader 210 can be detected by the processing device 200.

Hereinafter, the operation of the second embodiment will be described in detail.

Gestures (second gestures) in which the user inserts the card into the insertion slot 210a of the card reader 210 are patterned in advance, and are collected as a database in the user operation pattern storage unit 506 of the operation device identification unit 302 of FIG. Here, the gesture to be operated (second gesture) is, as described above, a pattern of the movement of the user trying to insert the card into the insertion slot 210a of the card reader 210.

In this way, as shown in FIG. 6, an operation pattern (second gesture) in which the user operates the card reader 210 is stored in advance in the user operation pattern storage unit 506 of the operation device identification unit 302. The failure determination control unit 306 is detected by the device output detection unit 304 and the user operation state detected by the user operation detection unit 504 based on the camera image captured by the camera 202 mounted on the processing device 200 shown in FIG. It is determined whether or not the user is operating the card reader 210 by comparing with the operating state of the device. When an error is detected, the failure determination control unit 306 issues an error notification to the error notification unit 310.

As described above, the failure determination control unit 306 includes a device operation determination unit 402, an operation device determination unit 404, a failure determination unit 406, and a recovery control unit 408.

In the second embodiment, the device operation determination unit 402 detects the device output detection signal supplied from the device output detection unit 304, which indicates that the output signal has been output from the card reader 210, and the card reader 210 detects the device output detection signal. Determine if it is working. The operation device determination unit 404 determines that the device being operated is the card reader 210 from the user operation detection result signal detected by the user operation detection unit 504. The failure determination unit 406 compares the operating state of the card reader with the operating state of the user, and determines the operating state of the card reader 210. The recovery control unit 408 outputs a reset signal for resetting the card reader 210 when the user operation state of the card reader 210 detected by the failure determination unit 406 and the card reader operation state do not match.

It is assumed that the camera 202 mounted on the processing device 200 shown in FIG. 2 acquires a camera image showing an operation status in which the user is operating the card reader 210. At this time, the user operation detection unit 504 refers to the user operation pattern stored in the user operation pattern storage unit 506, and outputs a user operation detection result signal indicating that the user is operating the card reader 210. In response to this user operation detection result signal, the operation device determination unit 404 determines that the operation target device is the card reader 210. The device output detection unit 304 monitors the output signal from the determined card reader 210, and the device operation determination unit 402 determines the operation state of the card reader 210. The failure determination unit 406 determines the operating state of the card reader 210 from the detected user operating state of the card reader 210 and the device operating state of the card reader 210. Thereby, the failure determination can be performed by the processing device 200 without the user's declaration. When the failure determination unit 406 determines that the user operation state of the card reader 210 and the device operation state of the card reader 210 match, the processing device 200 continues the normal operation. On the other hand, when the failure determination unit 406 determines that there is a mismatch, the recovery control unit 408 performs device recovery control. If they do not match a plurality of times, the failure determination unit 406 determines that an error occurs and issues an error message to the error notification unit 310.

Since the operation when the device is the numeric keypad 208 is the same as that of the first or second embodiment, the description thereof will be omitted.

The above-mentioned fault detection device 300 and fault detection device 300A may be realized by hardware or software. Further, the fault detection device 300 and the fault detection device 300A may be realized by a combination of hardware and software.

[Hardware configuration of fault detector]
FIG. 9 is a block diagram showing an example of hardware of an information processing device (computer) when the failure detection device 300 and the failure detection device 300A are realized by software.

As shown in FIG. 9, the information processing device 600 includes a control unit (CPU: Central Processing Unit) 610, a storage unit 620, a ROM (Read Only Memory) 630, a RAM (Random Access Memory) 640, and a communication interface. It includes a 650 and a user interface 660.

The control unit (CPU) 610 can realize various functions of the failure detection device 300 and the failure detection device 300A by expanding the program stored in the storage unit 620 or the ROM 630 into the RAM 640 and executing the program. Further, the control unit (CPU) 610 may include an internal buffer that can temporarily store data and the like.

The storage unit 620 is a large-capacity storage medium that can hold various types of data, and can be realized by a storage medium such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive). Further, the storage unit 620 may be a cloud storage existing on the communication network when the information processing device 600 is connected to the communication network via the communication interface 650. Further, the storage unit 620 may hold a program that can be read by the control unit (CPU) 610.

The ROM 630 is a non-volatile storage device that can be configured with a flash memory or the like having a smaller capacity than the storage unit 620. Further, the ROM 630 may hold a program that can be read by the control unit (CPU) 610. The program that can be read by the control unit (CPU) 610 may be held by at least one of the storage unit 620 and the ROM 630.

The program that can be read by the control unit (CPU) 610 may be supplied to the information processing device 600 in a state of being non-temporarily stored in various storage media that can be read by a computer. Such storage media include, for example, magnetic tapes, magnetic disks, optomagnetic disks, CD-ROMs (Compact Disc-Read Only Memory), CD-Rs (Compact Disc-Recordable), and CD-RWs (Compact Disc-ReWritable). , A semiconductor memory.

The RAM 640 is a semiconductor memory such as a DRAM (Dynamic Random Access Memory) and a SRAM (Static Random Access Memory), and can be used as an internal buffer for temporarily storing data or the like.

The communication interface 650 is an interface for connecting the information processing device 600 and the communication network via a wired or wireless connection.

The user interface 660 is, for example, a display unit such as a display and an input unit such as a keyboard, a mouse, and a touch panel.

Although the present invention has been described above with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention.

The present invention can also be used for processing devices such as cash processing devices, multimedia terminals, and store terminals (POS) that are operated by users.

200 Processing device 202 Camera 204 Visor 206 Touch panel display 208 Numeric keypad (device)
210 Card reader (device)
210a Insertion 212 Touch panel (device)
214 Display panel 216 Viewing angle limiting filter 300, 300A Fault detection device 301 CPU
302 Operation device identification unit 304 Device output detection unit 306 Failure judgment control unit 308 Device reset control unit 310 Error notification unit 402 Device operation judgment unit 404 Operation device judgment unit 406 Failure judgment unit 408 Recovery control unit 502 Camera control unit 504 User operation detection Unit 506 User operation pattern storage unit 600 Information processing device 610 Control unit (CPU)
620 Storage unit 630 ROM
640 RAM
650 communication interface 660 user interface

Claims (9)

A failure detection device that detects failures of multiple devices that can be operated by the user in the processing device.
A camera capable of capturing the operation status of the user with respect to the plurality of devices,
Before actually operating the processing device, an operation gesture in which the operation status of the user operating the plurality of devices is represented by a camera image taken by the camera in advance is patterned for each of the plurality of devices. A user operation pattern storage unit that stores a plurality of user operation patterns obtained in advance is included, and is stored in the user operation pattern storage unit based on a camera image taken by the camera during operation of the processing device. By referring to the plurality of user operation patterns, an operation device identification unit that identifies which device the user is operating and outputs a user operation detection result signal, and an operation device identification unit.
A device output detection unit that detects a signal output from any of the plurality of devices and outputs a device output detection signal.
A failure determination control unit that compares the device output detection signal with the user operation detection result signal to determine the presence or absence of failure of any of the plurality of devices.
A device failure detector. The operation device identification unit,
A camera control unit that controls the camera and acquires the camera image,
A user operation that identifies a device operated by the user by referring to the plurality of user operation patterns stored in the user operation pattern storage unit from the acquired camera image, and outputs the user operation detection result signal. With the detector
The device failure detection device according to claim 1. The device failure detection device according to claim 1 or 2, wherein the plurality of devices include at least two of a touch panel, a card reader, and a numeric keypad. The operation gesture is a first gesture representing a first operation status in which the user operates the touch panel, and a second gesture representing a second operation status in which the user inserts a card into the insertion slot of the card reader. The device failure detection device according to claim 3, further comprising at least two of a third gesture representing a third operation situation in which the user operates the numeric keypad. The failure determination control unit
A device operation determination unit that detects the device output detection signal, determines whether or not any of the plurality of devices is operating, and outputs a device operation determination result signal indicating the device operation state.
An operation device determination unit that determines a device operated by the user in response to the user operation detection result signal and outputs a user operation state determination result signal indicating the user operation state.
By comparing the device operation determination result signal with the user operation state determination result signal, it is determined whether or not the user operation state matches or does not match the device operation state, and any of the plurality of devices fails. A failure judgment unit that outputs a failure judgment result signal that indicates the presence / absence judgment result,
The device failure detection device according to any one of claims 1 to 4, wherein the device has. The failure determination control unit outputs a reset signal for recovering the device determined to be a failure when the failure determination unit determines that the user operation state and the device operation state do not match. The failure detection device for the device according to claim 5, further comprising a control unit. 5. The failure determination unit counts the number of times of the failure result determination signal indicating the determination result of failure, and issues an error message to the error notification unit when the counted number of times reaches a predetermined number of times. Or the failure detection device of the device according to 6. It is a failure detection method that detects failures of multiple devices that can be operated by the user in the processing device.
Before operating the processor actually, by the plurality of camera capable of photographing an operation status by the user to the device, the camera which the user has shooting Me beforehand the operation status of the operation to the plurality of devices A plurality of user operation patterns obtained by patterning the operation gesture represented by the image for each of the plurality of devices are stored in advance in the user operation pattern storage unit.
During the operation of the processing device, the user operates which device by referring to the plurality of user operation patterns stored in the user operation pattern storage unit based on the camera image taken by the camera. It identifies whether it is, and outputs the user operation detection result signal.
A signal output from any of the plurality of devices is detected, and a device output detection signal is output.
By comparing the device output detection signal with the user operation detection result signal, it is determined whether or not any of the plurality of devices has failed.
Device failure detection method. A failure detection program that causes a computer to detect failures in multiple devices that can be operated by a user in a processing device.
Before operating the processor actually, by the plurality of camera capable of photographing an operation status by the user to the device, the camera which the user has shooting Me beforehand the operation status of the operation to the plurality of devices A user operation pattern storage procedure for storing a plurality of user operation patterns obtained by patterning an operation gesture represented by an image for each of the plurality of devices in a user operation pattern storage unit in advance, and
During the operation of the processing device, the user operates which device by referring to the plurality of user operation patterns stored in the user operation pattern storage unit based on the camera image taken by the camera. The operation device identification procedure that identifies whether or not the camera is used and outputs the user operation detection result signal,
A device output detection procedure that detects a signal output from any of the plurality of devices and outputs a device output detection signal, and
A failure determination control procedure for determining the presence or absence of a failure of any of the plurality of devices by comparing the device output detection signal with the user operation detection result signal.
A failure detection program for causing the computer to execute.

Images