JP7834672B2 - Automated trading device - Google Patents

Description

This invention relates to an automated trading device.

Bank account holders may use automated teller machines (ATMs) to conduct transactions related to their bank accounts. ATMs may be equipped with cameras that record the operator's image, such as saving images of users engaging in fraudulent transactions, for security purposes.

Regarding technologies for photographing ATM operators, for example, an automated transaction device has been proposed that consistently and accurately captures customer images as evidence of fraudulent transactions. Furthermore, an automated transaction device has been proposed that guides the user to operate with both hands and has a function to release the camera window's obstruction without interrupting the transaction process.

Japanese Patent Publication No. 2004-139447 Japanese Patent Publication No. 2010-86386

If an ATM camera is blocked, it becomes impossible to film the operator, reducing its security effectiveness. Therefore, it is advisable to detect when the camera is blocked and notify administrators. One method of detecting camera blockage is to use a system where the entire captured image is black. However, this system might fail to detect camera blockage if only a portion of the camera is blocked.

One aspect of this project is its aim to improve crime prevention.

One proposed solution provides an automated trading device comprising a camera and a control unit. The camera is positioned to capture images of the operator. The control unit extracts a continuous region from the image captured by the camera, where the brightness is below a predetermined value. Based on whether the region includes pixels at the edge of the image and whether the region is greater than a threshold, the control unit determines whether the camera is blocked.

According to one embodiment, the crime prevention effect can be improved.

This figure shows an example of an information processing system according to the first embodiment. This figure shows an example of an information processing system according to the second embodiment. This diagram shows an example of an ATM hardware configuration. This is a diagram showing an example of the appearance of an ATM. This is a block diagram showing examples of ATM functions. This figure shows an example of image processing according to the second embodiment. This is a flowchart illustrating an example of the procedure for detecting when a camera is blocked.

The following description of this embodiment will be made with reference to the drawings. Note that each embodiment can be implemented by combining multiple embodiments within a reasonable scope.
[First Embodiment]
First, let me describe the first embodiment.

Figure 1 shows an example of an information processing system according to the first embodiment. In the first embodiment, the automated trading device 10 detects when the camera 11, which photographs the operator of the automated trading device 10, is blocked. The information processing system of the first embodiment includes a terminal 1 and an automated trading device 10.

Terminal 1 is a computer connected to an automated trading system (ATM). For example, Terminal 1 is a computer connected to the automated trading system 10 via a network and operated by the administrator of the automated trading system 10. The automated trading system 10 is a device that executes financial transactions according to the operator's instructions. The automated trading system 10 is, for example, an ATM installed in a financial institution. The automated trading system 10 includes a camera 11 and a control unit 12.

Camera 11 is a camera installed to photograph the operator of the automated trading device 10. For example, camera 11 is installed on the automated trading device 10 so that its lens is pointed towards the position where the operator of the automated trading device 10 is standing. The control unit 12 controls the automated trading device 10 and can execute the required processing. The control unit 12 is, for example, a processor or arithmetic circuit within the automated trading device 10.

First, the control unit 12 acquires the captured image 2 taken by the camera 11. For example, when the control unit 12 detects an operator operating the automated trading device 10 using a motion sensor or the like, it activates the camera 11 and has it record video of the operation until the operator leaves the automated trading device 10. The control unit 12 then acquires the captured image 2, which is the video of the operation.

The control unit 12 extracts a region 2a from the captured image 2 where pixels with a brightness below a predetermined value are continuous. The control unit 12 determines whether the numerical value indicating the degree of brightness of each pixel in the captured image 2 is below a predetermined value, and extracts a region 2a where pixels whose brightness level is determined to be below the predetermined value are continuous. The numerical value indicating the degree of brightness can be, for example, a pixel value, a gradation value, or a luminance value. For example, the control unit 12 converts the captured image 2 to grayscale and smooths it. The control unit 12 then performs a binarization process on the smoothed captured image 2, making pixels with a brightness level below a predetermined value black and pixels with a brightness level greater than a predetermined value white. Finally, the control unit 12 extracts a region 2a of continuous black pixels from the binarized captured image 2.

The control unit 12 then determines whether the camera 11 is blocked based on whether region 2a includes pixels at the edge of the captured image 2 and whether region 2a is greater than a threshold. Here, the control unit 12 determines whether the camera 11 is blocked based on whether region 2a includes all pixels on any of the four sides of the captured image 2. For example, if region 2a includes all pixels on any of the four sides of the captured image 2, the control unit 12 determines that region 2a includes pixels at the edge of the captured image 2. Furthermore, if region 2a includes all pixels on any of the four sides of the captured image 2 AND region 2a is greater than a threshold, the control unit 12 determines that the camera 11 is blocked.

If the control unit 12 determines that camera 11 is blocked, it notifies terminal 1 that camera 11 is blocked. For example, the control unit 12 sends a message to terminal 1 indicating that camera 11 is blocked.

According to the first embodiment, the camera 11 of the automated trading device 10 is installed to capture images of the operator. The control unit 12 of the automated trading device 10 extracts a region 2a from the captured image 2 taken by the camera 11, where pixels with a brightness below a predetermined value are continuous. The control unit 12 then determines whether the camera 11 is blocked based on whether the region 2a includes pixels at the edge of the captured image 2 and whether the region 2a is greater than a threshold.

Here, if camera 11 is obstructed by an object such as the operator's hand or belongings, the obstruction will appear in the captured image 2 from the direction in which the object approached camera 11. As a result, the edges of the captured image 2 become dark due to the obstruction. Therefore, the automated trading device 10 can appropriately determine whether camera 11 is obstructed by determining whether the dark pixel area starting from the edge is greater than a threshold. Thus, the automated trading device 10 can improve its security effectiveness.

Furthermore, the control unit 12 determines whether the camera 11 is blocked based on whether region 2a includes all the pixels on any of the four sides of the captured image 2. Here, since the obstruction blocking the camera 11 is often placed close to the camera 11, the entire edge in the direction in which the obstruction appears in the captured image 2 is often darkened. Therefore, the automated trading device 10 can improve the accuracy of determining whether the camera 11 is blocked by determining whether the camera 11 is blocked based on whether region 2a includes all the pixels on its edges.

Furthermore, the control unit 12 smooths the captured image 2 and extracts a region 2a from the captured image 2 where pixels with a brightness below a predetermined value are continuous. This allows the automated trading device 10 to remove noise from the captured image 2 and improve the accuracy of determining whether or not the camera 11 is blocked.

Furthermore, if the control unit 12 determines that camera 11 is blocked, it notifies terminal 1 that camera 11 is blocked. This allows the automated trading device 10 to notify the operator of terminal 1 that camera 11 is not properly capturing images of the operator.

[Second Embodiment]
Next, a second embodiment will be described. The second embodiment detects when a camera installed in an ATM is blocked.

Figure 2 shows an example of an information processing system according to the second embodiment. The information processing system of the second embodiment includes an ATM 100 and a terminal 31. The ATM 100 is a device that receives input operations from customers and executes financial transactions based on the received input. The ATM 100 is connected to the terminal 31 via a network 20. The network 20 is, for example, the LAN (Local Area Network) of the bank where the ATM 100 is installed. The terminal 31 is a computer operated by the administrator of the ATM 100.

When the ATM 100 is being operated by a user, the ATM 100 uses its camera to photograph the user. Based on the image captured by the ATM 100's camera, the ATM 100 determines whether the camera is blocked. If the ATM 100 detects that the camera is blocked, it notifies the terminal 31 that the camera is blocked.

Figure 3 shows an example of the hardware configuration of an ATM. The ATM 100 is controlled entirely by a processor 101. The processor 101 is connected to a memory 102 and several peripheral devices via a bus 108. The processor 101 may be a multiprocessor. The processor 101 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). At least some of the functions realized by the processor 101 executing programs may be realized by electronic circuits such as an ASIC (Application Specific Integrated Circuit) or a PLD (Programmable Logic Device).

Memory 102 is used as the main memory of the ATM 100. Memory 102 temporarily stores at least a portion of the OS (Operating System) program and application programs to be executed by the processor 101. Memory 102 also stores various data used for processing by the processor 101. A volatile semiconductor memory device, such as RAM (Random Access Memory), is used as memory 102.

Peripheral devices connected to bus 108 include a storage device 103, a network interface 104, a display processing unit 105, a touch panel processing unit 106, and an I/O (Input/Output) interface 107.

The storage device 103 electrically or magnetically writes and reads data from its built-in recording medium. The storage device 103 is used as an auxiliary storage device for the ATM 100. Various data, including OS programs, application programs, and transaction history information, are stored in the storage device 103. For example, an HDD (Hard Disk Drive) or SSD (Solid State Drive) can be used as the storage device 103.

The network interface 104 is connected to the network 20. The network interface 104 transmits and receives data to and from the terminal 31 via the network 20.

The display processing unit 105 is connected to the display 111. The display processing unit 105 displays various information, such as operation guides, on the screen of the display 111 according to instructions from the processor 101. The display 111 can be an organic EL (Electro-Luminescence) display device or a liquid crystal display device, among others.

The touch panel processing unit 106 is connected to the touch panel 112. The touch panel 112 is positioned, for example, in front of the display 111. The touch panel processing unit 106 detects the position on the screen where the user's finger touches or approaches the touch panel 112 and notifies the processor 101.

The I/O interface 107 is connected to a card processing unit 113, a passbook processing unit 114, a coin processing unit 115, a banknote processing unit 116, a human presence sensor 117, and a camera 118. The I/O interface 107 notifies each connected unit of instructions from the processor 101 according to the processor's commands. The I/O interface 107 also notifies the processor 101 of information acquired from each unit via the bus 108.

The card processing unit 113 controls the insertion and ejection of cards. The card processing unit 113 reads the information (account number, etc.) recorded on the magnetic strip attached to the inserted card.

The passbook processing unit 114 controls the insertion and ejection of passbooks. It also reads the magnetically recorded information (account number, etc.) attached to the inserted passbook. Furthermore, the passbook processing unit 114 has a function to print on the passbook, enabling passbook entries.

The coin handling unit 115 controls the dispensing and receiving of coins, and the opening and closing of the coin slot door, according to instructions from the processor 101. The banknote handling unit 116 controls the dispensing and receiving of banknotes, and the opening and closing of the banknote slot door, according to instructions from the processor 101. The banknote handling unit 116 has a function to read the type and serial number of banknotes. The banknote handling unit 116 reads the serial numbers of banknotes inserted and removed during deposits and withdrawals and outputs them to the processor 101.

The human presence sensor 117 detects a person and outputs the detection result to the processor 101. For example, when the human presence sensor 117 receives infrared light emitted from a human body within a predetermined range, it notifies the processor 101 that a human body has been detected. The human presence sensor 117 is, for example, an infrared sensor.

The camera 118 generates still image or video data of the scene pointed at its lens, according to instructions from the processor 101, and stores it in the memory 102. The camera 118 is positioned to photograph the ATM 100 operator.

Furthermore, the I/O interface 107 can read data from or write data to the portable recording medium 25. The portable recording medium 25 is, for example, a DVD (Digital Versatile Disc) or a CD (Compact Disc). The portable recording medium 25 can store transaction history information, etc.

Furthermore, the automated transaction device 10 shown in the first embodiment can also be implemented using the same hardware as the ATM 100 shown in Figure 3. Also, the processor 101 is an example of the control unit 12 shown in the first embodiment.

The ATM 100 implements the processing functions of the second embodiment by executing a program recorded on, for example, a computer-readable recording medium. The program describing the processing content to be executed by the ATM 100 can be recorded on various recording media. For example, the program to be executed by the ATM 100 can be stored in the storage device 103. The processor 101 loads at least a portion of the program in the storage device 103 into the memory 102 and executes the program. Alternatively, the program to be executed by the ATM 100 can be recorded on the portable recording medium 25. The program stored on the portable recording medium 25 becomes executable after being installed in the storage device 103, for example, under control from the processor 101. The processor 101 can also directly read and execute the program from the portable recording medium 25.

Next, I will describe the appearance of ATM100.
Figure 4 shows an example of the appearance of an ATM. The ATM 100 has a horizontal first surface and a second surface positioned in front of the operator. A display 111 and a touch panel 112 are positioned on the first surface. The display 111 shows the operation screen of the ATM 100. The touch panel 112 is positioned in front of the display 111. The touch panel 112 detects the operator's touch operation on the screen displayed by the display 111. As a result, the display 111 and the touch panel 112 provide guidance on transaction operations by the ATM 100 and receive instructions from the operator.

The second side features a card slot 21, a passbook slot 22, a banknote slot 23, a coin slot 24, a motion sensor 117, and a camera 118. The card slot 21 is used for inserting a cash card by the operator and for ejecting the cash card by the card processing unit 113. The card processing unit 113 reads information such as the account number from the cash card inserted into the card slot 21. The card processing unit 113 also ejects the cash card from the card slot 21 after the transaction is completed.

The passbook slot 22 is used for inserting passbooks by the operator and for ejecting passbooks by the passbook processing unit 114. The passbook processing unit 114 reads information such as the account number from the passbook inserted into the passbook slot 22. The passbook processing unit 114 also prints information onto the passbook inserted through the passbook slot 22. Finally, the passbook processing unit 114 ejects the passbook from the passbook slot 22 after the transaction is completed.

The banknote slot 23 is used for inserting banknotes by the operator and for dispensing banknotes by the banknote processing unit 116. The banknote processing unit 116 reads the banknotes inserted into the banknote slot 23. The banknote processing unit 116 also dispenses the dispensed banknotes from the banknote slot 23. The coin slot 24 is used for inserting coins by the operator and for dispensing coins by the coin processing unit 115. The coin processing unit 115 reads the coins inserted into the coin slot 24. The coin processing unit 115 also dispenses the dispensed coins from the coin slot 24.

The motion sensor 117 is installed so that its light-receiving section faces the front of the second surface. When an operator approaches a position where they can operate the touch panel 112, the motion sensor 117 receives infrared light emitted by the operator and can detect the operator.

Camera 118 is positioned so that its lens faces the front of the second surface. While the motion sensor 117 detects an operator, camera 118 photographs the operator at the operating position. The ATM 100 then stores the video captured by camera 118 in the storage device 103.

In this way, the ATM 100 can photograph the operator using the camera 118. For example, the ATM 100 can save the captured video and, if a fraudulent transaction occurs, provide the video of that transaction to identify the operator who committed the fraud. However, if the camera 118 is blocked, the ATM 100 cannot properly photograph the operator. Therefore, the ATM 100 detects that the camera 118 is blocked based on the image captured by the camera 118 and notifies the terminal 31 operated by the administrator. Next, the functions of the ATM 100 will be explained in detail.

Figure 5 is a block diagram illustrating an example of ATM functionality. The ATM 100 includes an image processing unit 120, a determination unit 130, and a notification unit 140. The image processing unit 120, the determination unit 130, and the notification unit 140 are implemented by the processor 101 executing a program stored in the memory 102.

The image capture processing unit 120 controls the camera 118 to acquire images of the ATM 100 operator. When the human presence sensor 117 detects an ATM 100 operator, the image capture processing unit 120 instructs the camera 118 to begin capturing images. The image capture processing unit 120 then controls the camera 118 to continue capturing images until the human presence sensor 117 no longer detects an ATM 100 operator. The image capture processing unit 120 then acquires the images captured by the camera 118.

The determination unit 130 processes the captured image acquired by the image processing unit 120 and determines whether or not the camera 118 is blocked. The determination unit 130 converts the captured image acquired from the camera 118 to grayscale. The determination unit 130 smooths the grayscale image. The determination unit 130 then binarizes the smoothed image. For example, the determination unit 130 generates an image where pixels with a gradation value below a predetermined value are black, and pixels with a gradation value greater than the predetermined value are white. The determination unit 130 then extracts regions where black pixels are continuous in the binarized image.

The determination unit 130 then determines whether the camera 118 is blocked based on whether the extracted region contains all the pixels on any edge of the image and whether the area of the extracted region is greater than a threshold. The determination unit 130 determines that the camera 118 is blocked if the extracted region contains all the pixels on any edge of the image AND the area of the extracted region is greater than a threshold.

If the determination unit 130 determines that the camera 118 is blocked, the notification unit 140 notifies the terminal 31 that the camera 118 is blocked. For example, the notification unit 140 sends a message to the terminal 31 indicating that the camera 118 is blocked.

Note that the lines connecting each element shown in Figure 5 represent only a portion of the communication path; other communication paths can also be configured. Next, the image processing for determining whether or not camera 118 is blocked will be explained in detail.

Figure 6 shows an example of image processing in the second embodiment. The determination unit 130 performs image processing on the image captured by the camera 118 of the ATM 100 operator and determines whether or not the camera 118 is blocked.

First, the determination unit 130 converts the captured image from the camera 118 to grayscale to generate image 41. Image 41 is an image in which the RGB values of each pixel in the captured image have been converted to tone values. Note that a higher tone value indicates a brighter pixel. Next, the determination unit 130 smooths and binarizes image 41 to generate image 42. In smoothing, the determination unit 130 combines a predetermined number of pixels into one pixel and reduces the number of tone values (the range of tone values) to reduce variations in density. For example, the determination unit 130 combines 8x8 pixels of image 41 into one pixel and reduces the number of tone values from 256 to 8. In binarization, the determination unit 130 treats pixels with tone values below a predetermined value in the smoothed image 41 as black, and pixels with tone values greater than the predetermined value as white.

Image 42 is an image obtained by the determination unit 130 from the captured image, where each pixel is either black or white. The determination unit 130 determines whether the camera 118 is blocked based on the distribution of black pixels in image 42. The determination unit 130 extracts a region from image 42 where black pixels are continuous. The determination unit 130 then determines whether the extracted region contains all pixels on any of the edges of image 42 and whether the extracted region is greater than a threshold. Note that "pixels on any of the edges" refers to the pixels at the very edge (left, right, top, or bottom).

The determination unit 130 determines that the camera 118 is blocked if the extracted region contains all pixels on any edge of the image 42 and the extracted region is greater than the threshold. The determination unit 130 also determines that the camera 118 is not blocked if the extracted region does not contain any pixels on any edge of the image 42, or if the extracted region is less than or equal to the threshold. Furthermore, if there are multiple regions with continuous black pixels, the determination unit 130 performs the above determination on all of these regions, and if it determines that the camera 118 is blocked in any of these regions, it determines that the camera 118 is blocked.

In this way, the determination unit 130 determines whether or not the camera 118 is blocked. When the camera 118 is blocked by an obstruction such as the operator's hand or belongings, the obstruction will appear in the captured image from the direction in which the obstruction approached the camera 118. Furthermore, since the obstruction blocking the camera 118 is often placed close to the camera 118, the entire edge of the image in the direction in which the obstruction appears is often darkened. Therefore, the determination unit 130 extracts a region of continuous dark pixels, and determines that the camera 118 is blocked if all pixels on any edge of the captured image are included in the extracted region and the extracted region is greater than a threshold. This allows the determination unit 130 to appropriately determine whether or not the camera 118 is blocked.

One possible method for determining whether camera 118 is blocked is to determine if camera 118 is blocked when the entire captured image is dark. However, if only a part of camera 118 is blocked, the method of determining camera 118 is blocked when the entire captured image is dark cannot determine if camera 118 is blocked. On the other hand, in the second embodiment, the determination unit 130 determines whether camera 118 is blocked based on whether the area of dark pixels starting from the edge is greater than a threshold, so it can make an appropriate determination even when only a part of camera 118 is blocked. Therefore, the determination unit 130 can improve the security effect.

Furthermore, the determination unit 130 smooths the grayscale captured image, then binarizes it to extract regions where black pixels are continuous. By smoothing, the determination unit 130 reduces variations in the density of the captured image and removes noise. As a result, even if pixels on the edge where an obstruction is visible are brightened by noise, the determination unit 130 can remove the noise and extract the region, determining that all pixels on that edge are included in the extracted region. Therefore, by smoothing, the determination unit 130 can improve the accuracy of determining whether or not the camera 118 is blocked.

Next, we will explain in detail the camera obstruction detection process performed by ATM 100.
Figure 7 is a flowchart showing an example of the procedure for detecting camera obstruction. The process shown in Figure 7 will be explained below according to the step numbers.

[Step S11] The image processing unit 120 determines whether the human presence sensor 117 has detected an operator of the ATM 100. If the image processing unit 120 determines that the human presence sensor 117 has detected an operator of the ATM 100, it proceeds to step S12. If the image processing unit 120 determines that the human presence sensor 117 has not detected an operator of the ATM 100, it proceeds to step S11.

[Step S12] The image processing unit 120 instructs the camera 118 to start shooting and controls the camera 118 to continue shooting until the human presence sensor 117 no longer detects an operator of the ATM 100. The image processing unit 120 acquires the image captured by the camera 118.

[Step S13] The determination unit 130 converts the captured image acquired from the camera 118 into grayscale. For example, the determination unit 130 generates an image 41 by converting the RGB values of each pixel of the captured image acquired in step S12 into grayscale values.

[Step S14] The determination unit 130 smooths the captured image converted to grayscale in step S13. For example, the determination unit 130 consolidates a predetermined number of pixels in the image 41 into one pixel, reducing the number of gradations, which is the range of gradation values.

[Step S15] The determination unit 130 binarizes the image smoothed in step S14. For example, the determination unit 130 generates an image 42 in which pixels with a grayscale value less than or equal to a predetermined value are black, and pixels with a grayscale value greater than the predetermined value are white.

[Step S16] The determination unit 130 extracts a region where black pixels are continuous in the binarized image in step S15.
[Step S17] The determination unit 130 determines whether the region extracted in step S16 contains all the pixels on any of the edges of the image. If the determination unit 130 determines that the extracted region contains all the pixels on any of the edges of the image, it proceeds to step S18. If the determination unit 130 determines that the extracted region does not contain any pixels on any of the edges of the image, it proceeds to step S11.

[Step S18] The determination unit 130 determines whether the area of the region extracted in step S16 is greater than the threshold. If the determination unit 130 determines that the area of the extracted region is greater than the threshold, the process proceeds to step S19. If the determination unit 130 determines that the area of the extracted region is less than or equal to the threshold, the process proceeds to step S11.

[Step S19] The notification unit 140 notifies the terminal 31 that the camera 118 is blocked. For example, the notification unit 140 sends a message to the terminal 31 indicating that the camera 118 is blocked. Then, the process proceeds to step S11.

In this way, the ATM 100 extracts a continuous region of pixels with a brightness below a predetermined value from the image captured by the camera 118. Based on whether the extracted region includes pixels at the edge of the image and whether it is greater than a threshold, it determines whether the camera 118 is blocked. This allows the ATM 100 to appropriately determine whether the camera 118 is blocked. Therefore, the ATM 100 can improve its security effectiveness.

Furthermore, the ATM 100 determines whether the camera 118 is blocked based on whether the extracted region contains all the pixels on any of the four sides of the captured image. Since obstructions blocking the camera 118 are often placed close to the camera, the entire edge in the direction the obstruction appears in the captured image is often darkened. Therefore, the ATM 100 can improve its determination accuracy by determining whether the camera 118 is blocked based on whether the extracted region contains all the pixels on the edge. Additionally, the ATM 100 smooths the captured image and extracts regions with continuous black pixels from the binarized image obtained from the smoothed image. This allows the ATM 100 to improve its determination accuracy by removing noise from the captured image.

Furthermore, if ATM 100 determines that camera 118 is blocked, it notifies terminal 31 that camera 118 is blocked. This allows ATM 100 to notify its administrator that camera 118 is not properly capturing images of the user.

The above examples illustrate embodiments, but the configurations of each part shown in the embodiments can be replaced with others having similar functions. Furthermore, other arbitrary components or processes may be added. Moreover, any two or more configurations (features) from the embodiments described above may be combined.

1 Terminal 2 Captured image 2a Area 10 Automated trading device 11 Camera 12 Control unit

Claims (3)

A camera positioned to photograph the operator,
A control unit extracts a region from the image captured by the camera in which pixels with a brightness of less than or equal to a predetermined value are continuous, and determines whether the camera is blocked based on whether the region includes all pixels on any of the four sides of the image and whether the region is greater than a threshold.
An automated trading device having the following features. The control unit smooths the captured image and extracts a region from the captured image in which pixels with a brightness of less than or equal to a predetermined value are continuous.
The automated trading device according to claim 1. If the control unit determines that the camera is blocked, it notifies the terminal that the camera is blocked.
The automated trading device according to claim 1.