JP7623845B2 - Monitoring device, monitoring system, and fuel permit issuing terminal - Google Patents

Description

The present invention relates to a monitoring device, a monitoring system, and a refueling permission issuing terminal.

One type of gas station is the self-service gas station. Self-service gas stations are called "self-service stations." At self-service stations, customers fill up their tanks themselves. Customers are permitted to fill up their tanks under the supervision of a hazardous materials handler.

More specifically, a self-service station is provided with a monitoring room, and a self-service console (SSC) is installed inside the monitoring room. The staff at the self-service station directly and visually monitors customers at the station from the monitoring room, and when they are sure it is safe, they press the refueling permission switch on the SSC. As a result, the customer is permitted to refuel. Alternatively, a surveillance camera that captures images of the area around the weighing machine may be installed at the self-service station. In this case, the staff at the self-service station monitors customers at the station via images captured by the surveillance camera.

Patent Document 1 discloses a handheld transceiver equipped with a monitor and a refueling permission switch. The handheld transceiver can be carried by employees of a self-service gas station. Images captured by a surveillance camera are displayed on the monitor of the handheld transceiver. With the handheld transceiver of Patent Document 1, employees can monitor customers via the monitor of the handheld transceiver, and when safety is confirmed, they can press the refueling permission switch on the handheld transceiver to permit the customer to refuel.

JP 2002-193399 A

However, simply having employees of a self-service gas station directly and visually monitor customers from a monitoring room or through images captured by a surveillance camera does not mean that it is possible that a customer may be permitted to refuel even if a safety check is overlooked.

The present invention was made in consideration of the above problems, and its purpose is to provide a monitoring device, monitoring system, and refueling permission issuing terminal that can more reliably confirm the safety of refueling actions by customers at refueling stations themselves.

The monitoring device according to the present invention is communicatively connected to a refueling permission issuing terminal that accepts operations to permit a refueling operation by a refueling worker. The monitoring device includes an image receiving unit, an area extraction unit, an image generating unit, and a display image transmitting unit. The image receiving unit receives an image from an imaging device that captures an image of the periphery of a weighing machine installed at a refueling station. The area extraction unit extracts, from the image, an area requiring confirmation that needs to be confirmed in order to permit the refueling operation by the refueling worker. The image generating unit generates a display image including the image and a presentation image that presents the area requiring confirmation. The display image transmitting unit transmits the display image to the refueling permission issuing terminal.

The monitoring system according to the present invention comprises a monitoring device and an imaging device. The monitoring device is communicatively connected to a refueling permission issuing terminal that accepts operations to permit a refueling operation by a refueling worker. The imaging device captures an image of the periphery of a weighing machine installed at a refueling station. The monitoring device comprises an image receiving unit, an area extraction unit, an image generating unit, and a display image transmitting unit. The image receiving unit receives an image from the imaging device. The area extraction unit extracts, from the image, an area requiring confirmation that needs to be confirmed in order to permit the refueling operation by the refueling worker. The image generating unit generates a display image including the image and a presentation image that presents the area requiring confirmation. The display image transmitting unit transmits the display image to the refueling permission issuing terminal.

The above monitoring system may further include the refueling permission issuing terminal.

Another monitoring system according to the present invention includes a refueling permission issuing terminal and a monitoring device. The refueling permission issuing terminal accepts an operation for permitting a refueling action by a refueling worker. The monitoring device is communicably connected to the refueling permission issuing terminal. The monitoring device includes an image receiving unit, an area extracting unit, an image generating unit, and a display image transmitting unit. The image receiving unit receives an image from an imaging device that captures an image of the periphery of a weighing machine installed at a refueling station. The area extracting unit extracts a confirmation-required area from the image, which needs to be confirmed in order to permit the refueling action by the refueling worker. The image generating unit generates a display image including the image and a presentation image that presents the confirmation-required area. The display image transmitting unit transmits the display image to the refueling permission issuing terminal.

The refueling permission issuing terminal according to the present invention includes a display image receiving unit, a display unit, a decision unit, and a first operation acceptance unit. The display image receiving unit receives a display image from a monitoring device, the display image including an image acquired from an imaging device that captures an image of the periphery of a weighing machine installed at a refueling station, and a presentation image that presents a confirmation-required area that needs to be confirmed in order to permit the refueling action by the refueling worker. The display unit displays the display image. The decision unit decides whether or not to enable a refueling permission operation for permitting the refueling action by the refueling worker. The first operation acceptance unit accepts the refueling permission operation by an operator.

The monitoring device, monitoring system, and refueling permit issuing terminal according to the present invention can more reliably confirm the safety of refueling actions performed by customers at refueling stations themselves.

FIG. 1 illustrates a gas station system. FIG. FIG. 2 is a block diagram showing a configuration of a video processing server according to the first embodiment of the present invention. FIG. FIG. 4 is a diagram showing an example of a captured image. 1 is a block diagram showing a configuration of a mobile terminal according to a first embodiment of the present invention; FIG. 4 is a diagram showing a screen displayed on the mobile terminal according to the first embodiment of the present invention. FIG. 4 is a diagram showing a screen displayed on the mobile terminal according to the first embodiment of the present invention. FIG. 4 is a diagram showing a screen displayed on the mobile terminal according to the first embodiment of the present invention. 5 is a flowchart showing a process executed by a processing unit of the video processing server according to the first embodiment of the present invention. 5 is a flowchart showing a process executed by a terminal processing unit of the mobile terminal according to the first embodiment of the present invention. FIG. 11 is a diagram showing a screen displayed on a mobile terminal according to a second embodiment of the present invention. 10 is a flowchart showing a process executed by a processing unit of a video processing server according to a second embodiment of the present invention. 10 is a flowchart showing a process executed by a terminal processing unit of a mobile terminal according to a second embodiment of the present invention. FIG. 11 is a diagram showing a screen displayed on a mobile terminal according to another embodiment of the present invention.

Below, embodiments of the monitoring device, monitoring system, and fueling permission issuing terminal of the present invention will be described with reference to the drawings (Figs. 1 to 15). However, the present invention is not limited to the following embodiments, and can be implemented in various forms without departing from the gist of the invention. Note that where explanations overlap, they may be omitted as appropriate. Also, in the drawings, the same or equivalent parts are given the same reference symbols, and the same explanations will not be repeated.

[Embodiment 1]
A first embodiment of the present invention will be described with reference to Figs. 1 to 11. First, a gas station system 100 will be described with reference to Fig. 1. Fig. 1 is a diagram showing the gas station system 100. The gas station system 100 is applied to a self-service gas station (a self-service gas station). The gas station system 100 is a system that allows customers of the self-service gas station to refuel themselves. Hereinafter, a customer of the self-service gas station may be referred to as a "customer". A customer is, for example, a passenger in a vehicle C (fuel target) such as a passenger car or a motorcycle.

As shown in FIG. 1, the gas station system 100 includes a monitoring system 2, a weighing machine 4, an SSC (self-service console) 6, and a POS (point of sale) 8. The weighing machine 4 is installed on the premises of the self-service stand SS. The SSC 6 and the POS 8 are installed in a monitoring room of the self-service stand SS. Note that although there is only one weighing machine 4 in this embodiment, the gas station system 100 may include multiple weighing machines 4.

The metering machine 4 is a facility (fueling facility) that allows a customer to refuel. The metering machine 4 includes a fuel supply system and a control unit that controls the fuel supply system. The fuel supply system includes a fuel supply pump and a flow meter. The metering machine 4 includes a fuel supply nozzle 41 (see FIG. 2), and when the fuel supply pump is activated, fuel is supplied to the vehicle C from the fuel supply nozzle 41.

The weighing machine 4 is connected to the SSC 6 via wire or wirelessly so that it can communicate with the SSC 6. The weighing machine 4 accepts the customer's selection of the type of oil, the selection of the payment method, and the selection of the amount of fuel to be supplied to the vehicle C or the payment amount. The weighing machine 4 transmits data indicating the type of oil, the payment method, and the amount of fuel to be supplied or the payment amount selected by the customer to the SSC 6.

The POS 8 is connected to the SSC 6 via wire or wirelessly so that it can communicate with the SSC 6. The POS 8 collects data indicating the type of fuel selected by the customer, the payment method, and the amount of fuel dispensed or the payment amount via the SSC 6, and performs settlement processing and sales management. In this embodiment, the gas station system 100 includes the SSC 6 and the POS 8, but the gas station system 100 may also include a POS that includes the SSC 6.

The monitoring system 2 is a system for monitoring the refueling actions of customers (refueling workers). As shown in FIG. 1, the monitoring system 2 includes a video processing server 22, a plurality of mobile terminals 24, and a plurality of imaging devices 26. The video processing server 22 is an example of a monitoring device, and the mobile terminals 24 are an example of a refueling permission issuing terminal.

The imaging device 26 captures an image of the periphery of the weighing machine 4 to generate an image T. The imaging device 26 is communicatively connected to the image processing server 22. For example, the imaging device 26 may be communicatively connected to the image processing server 22 via a public communication network such as the Internet. Alternatively, the imaging device 26 may be communicatively connected to the image processing server 22 by wire or wirelessly. The imaging device 26 transmits the image T to the image processing server 22.

In this embodiment, the multiple imaging devices 26 include a first imaging device 26a and a second imaging device 26b. However, the number of imaging devices 26 is not limited to two. The surveillance system 2 may include three or more imaging devices 26. Alternatively, the number of imaging devices 26 may be one.

The video processing server 22 generates a display image to be displayed on the mobile terminal 24 based on the captured image T (an image showing the area around the weighing machine 4). The video processing server 22 further extracts a confirmation-required area from the captured image T. The confirmation-required area is an area that needs to be confirmed in order to allow the customer to fill up with fuel. The display image includes the captured image T and a first presentation image PR1. The first presentation image PR1 is an image that presents the confirmation-required area in the captured image T.

The areas requiring confirmation include, for example, a vehicle area, a person area, and a nozzle area. The vehicle area is the area where the vehicle C to be refueled is parked. The person area is the area where the customer performs the act of refueling. The nozzle area is the area where the nozzle portion of the refueling nozzle 41 (see FIG. 2) is inserted into the fuel filler opening of the vehicle C. The areas requiring confirmation are determined in advance.

The mobile terminal 24 is a terminal carried and used by an employee of the self-service station SS. Hereinafter, an employee of the self-service station SS may be referred to as an "employee." The employee can operate the mobile terminal 24 to authorize a customer to refuel. In other words, the mobile terminal 24 accepts an operation to authorize a customer to refuel. Hereinafter, an operation to authorize a customer to refuel may be referred to as a "refueling authorization operation."

The mobile terminal 24 is communicatively connected to the video processing server 22. For example, the mobile terminal 24 may be communicatively connected to the video processing server 22 via a public communication network such as the Internet. Alternatively, the mobile terminal 24 may be communicatively connected to the video processing server 22 wirelessly.

The mobile terminal 24 receives the display image (captured image T and first presentation image PR1) from the video processing server 22. The mobile terminal 24 displays the display image received from the video processing server 22.

In this embodiment, the mobile terminal 24 displays the first presentation image PR1 (an image presenting an area requiring confirmation within the captured image T). Therefore, the area requiring confirmation can be presented to the employee, making it less likely that the employee will overlook the area requiring confirmation. As a result, the safety of the customer's own refueling action can be more reliably confirmed.

The mobile terminal 24 is wirelessly connected to the SSC 6 so that it can communicate with the SSC 6. When the mobile terminal 24 receives a refueling permission operation, it transmits a refueling permission signal to the SSC 6. The refueling permission signal is a signal indicating that the refueling action is permitted.

In the example shown in FIG. 1, the monitoring system 2 includes two mobile terminals 24, but the number of mobile terminals 24 is not limited to two. The monitoring system 2 may include three or more mobile terminals 24. Alternatively, the number of mobile terminals 24 may be one.

The SSC 6 monitors the operation status of the weighing machine 4. In addition, the SSC 6 permits the customer to refuel in response to receiving a refueling permission signal from the mobile terminal 24. In other words, the customer cannot refuel until the SSC 6 permits the refueling.

In detail, the SSC 6 transmits a drive permission signal to the weighing machine 4 in response to receiving a refueling permission signal from the mobile terminal 24. The drive permission signal is a signal that permits the operation of the refueling system of the weighing machine 4. Specifically, the drive permission signal permits the operation of the refueling pump. Therefore, the customer can perform the act of refueling after the refueling permission signal is transmitted from the mobile terminal 24 to the SSC 6 and the drive permission signal is transmitted from the SSC 6 to the weighing machine 4.

Next, the gas station system 100 will be further described with reference to FIG. 1. The metering machine 4 transmits a refueling permission request signal to the SSC 6 when operated by a customer. The refueling permission request signal is a signal requesting permission for the customer to refuel. In detail, the metering machine 4 transmits a refueling permission request signal to the SSC 6 in response to receiving an operation to select the type of oil, an operation to select the payment method, and an operation to select the amount of fuel to be refueled or the payment amount.

In response to receiving a refueling permission request signal from the weighing machine 4, the SSC 6 transmits a monitoring start signal to the mobile terminal 24. In response to receiving the monitoring start signal, the mobile terminal 24 transmits a request signal requesting a display image to the video processing server 22. In response to receiving the request signal, the video processing server 22 transmits (distributes) the display image (captured image T and first presentation image PR1) to the multiple mobile terminals 24. As a result, the multiple mobile terminals 24 display the display image.

When the mobile terminal 24 accepts an operation performed by an employee on the display image, it transmits operation information indicating the content of the operation performed on the display image to the video processing server 22. Based on the operation information received from the mobile terminal 24, the video processing server 22 determines whether a predetermined operation has been performed on the area to be confirmed in the captured image T. If a predetermined operation has been performed on the area to be confirmed in the captured image T, the video processing server 22 executes a fixation process. The fixation process will be described later with reference to FIG. 4.

Next, the weighing machine 4 will be described with reference to FIG. 2. FIG. 2 is a front view of the weighing machine 4. As shown in FIG. 2, the weighing machine 4 includes fuel supply nozzles 41a, 41b, and 41c, fuel supply hoses 42a, 42b, and 42c, nozzle hangers 43a, 43b, and 43c, a touch display 44, and an antistatic sheet 45. The fuel supply nozzles 41a, 41b, and 41c, the fuel supply hoses 42a, 42b, and 42c, the nozzle hangers 43a, 43b, and 43c, the touch display 44, and the antistatic sheet 45 are provided on the front surface 4a of the weighing machine 4.

The metering machine 4 is equipped with a fuel supply system for each type of fuel: regular gasoline, high-octane gasoline, and diesel, in order to selectively supply the three types of fuel, regular gasoline, high-octane gasoline, and diesel, to the vehicle C.

One end of the refueling hose 42a is connected to a refueling system for regular gasoline. The other end of the refueling hose 42a is connected to the refueling nozzle 41a. Similarly, one end of the refueling hose 42b is connected to a refueling system for premium gasoline. The other end of the refueling hose 42b is connected to the refueling nozzle 41b. One end of the refueling hose 42c is connected to a refueling system for diesel. The other end of the refueling hose 42c is connected to the refueling nozzle 41c.

When the fuel nozzle 41a is not being used for fueling, it is hung on the nozzle hanger 43a. Similarly, when the fuel nozzle 41b is not being used for fueling, it is hung on the nozzle hanger 43b. When the fuel nozzle 41c is not being used for fueling, it is hung on the nozzle hanger 43c.

Hereinafter, when there is no need to distinguish between the refueling nozzles 41a to 41c, the refueling nozzles 41a to 41c may be referred to as "refueling nozzles 41". Similarly, when there is no need to distinguish between the refueling hoses 42a to 42c, the refueling hoses 42a to 42c may be referred to as "refueling hose 42". When there is no need to distinguish between the nozzle hangers 43a to 43c, the nozzle hangers 43a to 43c may be referred to as "nozzle hangers 43". Also, when there is no need to distinguish between the regular gasoline fuel system, the premium gasoline fuel system, and the diesel fuel system, the regular gasoline fuel system, the premium gasoline fuel system, and the diesel fuel system may be referred to as "fuel systems".

When a customer fills a vehicle C with fuel, the customer inserts the nozzle portion of the fuel nozzle 41 into the fuel inlet of the vehicle C and then pulls the lever of the fuel nozzle 41. At this time, if the metering machine 4 has already received the drive permission signal described with reference to FIG. 1 from the SSC 6, the fuel pump is activated and fuel is supplied from the fuel nozzle 41 to the fuel tank of the vehicle C. For example, when a customer fills a vehicle C with regular gasoline, the customer inserts the nozzle portion of the fuel nozzle 41a into the fuel inlet of the vehicle C and then pulls the lever of the fuel nozzle 41a. At this time, if the metering machine 4 has already received the drive permission signal from the SSC 6, the control unit of the metering machine 4 activates the fuel pump provided in the regular gasoline supply system. As a result, regular gasoline is supplied from the fuel nozzle 41a to the fuel tank of the vehicle C.

The touch display 44 accepts touch operations by the customer. The touch operation is, for example, a single tap. For example, the touch display 44 sequentially displays an oil type selection screen, a payment method selection screen, and a fuel quantity or payment amount selection screen in response to the customer's touch operations.

The oil type selection screen is a screen that allows the customer to select one of the oil types: regular gasoline, high-octane gasoline, and diesel. When the touch display 44 is displaying the oil type selection screen, the customer can select one of the oil types: regular gasoline, high-octane gasoline, and diesel by a touch operation (oil type selection operation).

The payment method selection screen is a screen that allows the customer to select a payment method. For example, the payment method selection screen allows the customer to select whether to pay by cash or by credit card. When the touch display 44 displays the payment method selection screen, the customer can select whether to pay by cash or by credit card by a touch operation (payment method selection operation).

The fuel quantity or payment amount selection screen is a screen that allows the customer to select the fuel quantity or payment amount. When the touch display 44 displays the fuel quantity or payment amount selection screen, the customer can select the fuel quantity or payment amount by a touch operation (fuel quantity or payment amount selection operation).

In response to the customer's touch operation, the metering machine 4 transmits the fueling permission request signal described with reference to FIG. 1 to the SSC 6. Specifically, the metering machine 4 transmits the fueling permission request signal to the SSC 6 after the customer selects the type of fuel, either regular gasoline, premium gasoline, or diesel, by a touch operation, selects whether to pay by cash or credit card, and selects the amount of fuel to be dispensed or the payment amount by a touch operation.

The static electricity removal sheet 45 removes static electricity that has been charged to the customer (fueling worker). Specifically, the static electricity that has been charged to the customer is removed when the customer touches the static electricity removal sheet 45 with their hand. After touching the static electricity removal sheet 45 with their hand, the customer performs the act of refueling.

The oil type selection operation, the payment method selection operation, and the fuel amount or payment amount selection operation may be performed on a single screen. Alternatively, any two of the oil type selection operation, the payment method selection operation, and the fuel amount or payment amount selection operation may be performed on a single screen.

Next, the self-service station SS will be described with reference to FIG. 3. FIG. 3 is a diagram showing a self-service station SS. As shown in FIG. 3, the self-service station SS is provided with multiple fuel lanes L and an island LA.

The island LA rises from the site of the self-service station SS. The weighing machine 4 is installed on the island LA. Fueling lanes L are provided on both sides of the island LA. Each fueling lane L includes a fueling area P. The fueling area P is adjacent to the weighing machine 4.

A customer at a self-service station SS drives a vehicle C into one of a number of fueling lanes L and then parks the vehicle in a fueling area P. In this embodiment, a number of frame lines FL are drawn on the premises of the self-service station SS. More specifically, a frame line FL is drawn for each fueling area P. A customer parks a vehicle C within a frame line FL and then refuels the vehicle C. Note that the frame lines FL may be omitted.

In this embodiment, the multiple refueling lanes L include a first refueling lane L1 and a second refueling lane L2. The first imaging device 26a shown in FIG. 1 images the refueling area P of the first refueling lane L1. The second imaging device 26b shown in FIG. 1 images the refueling area P of the second refueling lane L2.

Hereinafter, the refueling area P of the first refueling lane L1 may be referred to as the "first refueling area P1." Similarly, the refueling area P of the second refueling lane L2 may be referred to as the "second refueling area P2." Furthermore, the captured image T transmitted from the first imaging device 26a to the video processing server 22 may be referred to as the "first captured image T1." Similarly, the captured image T transmitted from the second imaging device 26b to the video processing server 22 may be referred to as the "second captured image T2." The first captured image T1 indicates the first refueling area P1, and the second captured image T2 indicates the second refueling area P2.

Next, the weighing machine 4 will be further described with reference to FIG. 3. As shown in FIG. 3, the weighing machine 4 has a front surface 4a and a rear surface 4b. The front surface 4a of the weighing machine 4 faces the first refueling lane L1 (first refueling area P1), and the rear surface 4b of the weighing machine 4 faces the second refueling lane L2 (second refueling area P2). In this embodiment, the weighing machine 4 further includes a refueling nozzle 41, a refueling hose 42, a nozzle hanger 43, a touch display 44, and a static electricity removing sheet 45 provided on the rear surface 4b.

In the example shown in FIG. 3, one island LA is provided at the self-service station SS, but multiple islands LA may be provided at the self-service station SS. The number of weighing machines 4, the number of fuel lanes L, and the number of imaging devices 26 increase as the number of islands LA increases.

Next, the configuration of the video processing server 22 of this embodiment will be described with reference to FIG. 4. FIG. 4 is a block diagram showing the configuration of the video processing server 22 of this embodiment. As shown in FIG. 4, the video processing server 22 includes a communication unit 221, a processing unit 222, and a storage unit 223. The communication unit 221 is an example of a captured image receiving unit, a display image transmitting unit, an operation information receiving unit, and a request signal receiving unit.

The communication unit 221 controls communication between the multiple mobile terminals 24 and the multiple imaging devices 26 described with reference to FIG. 1. The communication unit 221 has, for example, a LAN (Local Area Network) board or a LAN module. Specifically, the communication unit 221 receives a captured image T from the imaging device 26. The communication unit 221 receives a request signal described with reference to FIG. 1 from the mobile terminal 24. The communication unit 221 transmits a display image (captured image T and first presentation image PR1) described with reference to FIG. 1 to the mobile terminal 24. The communication unit 221 receives operation information described with reference to FIG. 1 from the mobile terminal 24.

The storage unit 223 includes, for example, a semiconductor memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and a large-capacity storage medium such as a HDD (Hard Disk Drive). The storage unit 223 stores various computer programs executed by the processing unit 222. The storage unit 223 also stores various types of data. The storage unit 223 saves the first captured image T1 and the second captured image T2 received by the communication unit 221 from the first imaging device 26a and the second imaging device 26b.

The processing unit 222 executes various processes. The processing unit 222 has a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The processor executes various processes by executing a computer program stored in the storage unit 223. The processing unit 222 may further have an integrated circuit dedicated to image processing. The integrated circuit dedicated to image processing may be an integrated circuit such as an ASIC (Application Specific Integrated Circuit). The processing unit 222 is an example of a region extraction unit.

The processing unit 222 extracts the area to be confirmed, as described with reference to FIG. 1, from the captured image T stored in the memory unit 223. The processing unit 222 may extract the area to be confirmed, for example, by pattern matching. Alternatively, the processing unit 222 may extract the area to be confirmed by using a trained model. In other words, the processing unit 222 may extract the area to be confirmed by AI (Artificial Intelligence).

When extracting a region to be checked by pattern matching, the storage unit 223 stores multiple reference images in advance. The processing unit 222 identifies a region to be checked in the captured image T by performing pattern matching between the captured image T and the multiple reference images.

When extracting areas requiring confirmation using AI, the storage unit 223 stores a trained model in advance. The processing unit 222 inputs the captured image T to the trained model. As a result, the trained model identifies areas requiring confirmation in the captured image T. When extracting areas requiring confirmation using AI, the processing unit 222 may further include an NCU (Neural Network Processing Unit).

The trained model is generated by executing a machine learning algorithm. The machine learning algorithm is, for example, any one of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning. For example, the trained model includes a decision tree, a nearest neighbor method, a naive Bayes classifier, a support vector machine, or a neural network. For example, the neural network includes an input layer, one or more intermediate layers, and an output layer. Specifically, the neural network is a deep neural network (DNN), a recurrent neural network (RNN), or a convolutional neural network (CNN).

When the processing unit 222 receives a request signal transmitted from the mobile terminal 24 via the communication unit 221, it generates a display image. Then, it controls the communication unit 221 to transmit the display image (captured image T and first presentation image PR1) to the mobile terminal 24. The first presentation image PR1 includes, for example, a text image indicating the area to be confirmed. Alternatively, the first presentation image PR1 may include a mark image indicating the area to be confirmed. The processing unit 222 is an example of an image generation unit.

In detail, the SSC 6 transmits a monitoring start signal indicating the identification information of the refueling lane L to be monitored to the mobile terminal 24. The mobile terminal 24 that receives the monitoring start signal transmits a request signal indicating the identification information of the refueling lane L to be monitored to the video processing server 22. The processing unit 222 that receives the request signal generates a display image including a captured image T corresponding to the refueling lane L to be monitored. The processing unit 222 then controls the communication unit 221 to transmit the display image to multiple mobile terminals 24. Hereinafter, the refueling lane L to be monitored may be referred to as the "target lane LT."

In this embodiment, the monitoring start signal indicates the identification information of the first refueling lane L1 or the second refueling lane L2. When a customer touches the touch display 44 provided on the front surface 4a of the weighing machine 4, the weighing machine 4 transmits a refueling permission request signal indicating the identification information of the first refueling lane L1 to the SSC 6. As a result, a monitoring start signal indicating the identification information of the first refueling lane L1 is transmitted from the SSC 6 to the mobile terminal 24, and a request signal indicating the identification information of the first refueling lane L1 is transmitted from the mobile terminal 24 to the image processing server 22. Similarly, when a customer touches the touch display 44 provided on the back surface 4b of the weighing machine 4, the weighing machine 4 transmits a refueling permission request signal indicating the identification information of the second refueling lane L2 to the SSC 6. As a result, a monitoring start signal indicating the identification information of the second refueling lane L2 is transmitted from the SSC 6 to the mobile terminal 24, and a request signal indicating the identification information of the second refueling lane L2 is transmitted from the mobile terminal 24 to the image processing server 22.

When the processing unit 222 receives a request signal indicating the identification information of the first refueling lane L1, it generates a display image including the first captured image T1. Similarly, when the processing unit 222 receives a request signal indicating the identification information of the second refueling lane L2, it generates a display image including the second captured image T2.

In this embodiment, the processing unit 222 further executes a first judgment process, a second judgment process, and a fixing process. The first judgment process, the second judgment process, and the fixing process are described below. The processing unit 222 is an example of a judgment result information generating unit.

<First Determination Process>
The first judgment process is a process for judging whether or not the image of the area to be confirmed matches the refueling non-permission condition. The refueling non-permission condition is a condition that does not permit a customer to refuel. When the processing unit 222 receives a request signal via the communication unit 221, the processing unit 222 executes the first judgment process based on the captured image T stored in the memory unit 223. The processing unit 222 executes the first judgment process by, for example, pattern matching. Alternatively, the processing unit 222 executes the first judgment process by using a trained model. Hereinafter, the judgment result of the first judgment process may be referred to as the "first judgment result."

The first judgment process is executed for each area to be confirmed. The first judgment result indicates whether or not the image of the area to be confirmed is an image that matches the refueling non-permission conditions. Therefore, the first judgment result indicates whether or not a situation has occurred in which refueling cannot be permitted in the area to be confirmed. The processing unit 222 generates judgment result information based on the first judgment result. The processing unit 222 then controls the communication unit 221 to transmit the judgment result information to the mobile terminal 24. The communication unit 221 is an example of a judgment result information transmission unit. The judgment result information indicates whether or not a situation has occurred in which refueling cannot be permitted for each area to be confirmed. In other words, the judgment result information indicates whether or not refueling may be permitted for each area to be confirmed. The judgment result information is, for example, a flag.

In the following, among the areas requiring confirmation, areas where situations have occurred that make refueling actions impermissible may be referred to as "NG areas." Also, among the areas requiring confirmation, areas where situations have not occurred that make refueling actions impermissible may be referred to as "OK areas."

<Second Determination Process and Fixation Process>
1, the mobile terminal 24 transmits operation information (information indicating the content of an operation performed on a display image) to the video processing server 22. When the processing unit 222 receives the operation information via the communication unit 221, it executes a second determination process based on the operation information. The second determination process is a process for determining whether or not a predetermined operation has been performed on a confirmation-required area in the captured image T. Hereinafter, the determination result of the second determination process may be referred to as the "second determination result."

The processing unit 222 executes a fixing process based on the second judgment result. In detail, when a predetermined operation is performed on a confirmation-required area in the captured image T, the second judgment result indicates that confirmation of the confirmation-required area where the predetermined operation was performed has been performed. When the processing unit 222 determines that a predetermined operation has been performed on a confirmation-required area in the captured image T, it fixes the judgment result for the confirmation-required area where the predetermined operation was performed in the judgment result information to a judgment result indicating that the refueling action may be permitted. Therefore, the judgment result information after the fixing process indicates that the confirmation-required area where the predetermined operation was performed is an OK area. The processing unit 222 controls the communication unit 221 to transmit the judgment result information after the fixing process to the mobile terminal 24.

Next, the confirmation-required areas will be described with reference to Figures 4 and 5. Figure 5 is a diagram showing an example of the captured image T. As shown in Figure 5, the processing unit 222 identifies, for example, confirmation-required areas DA1 to DA3 in the captured image T. In this embodiment, the confirmation-required area DA1 is a vehicle area. The confirmation-required area DA2 is a person area. The confirmation-required area DA3 is a nozzle area. Hereinafter, the confirmation-required area DA1 may be referred to as the "vehicle area DA1". Similarly, the confirmation-required area DA2 may be referred to as the "person area DA2". The confirmation-required area DA3 may be referred to as the "nozzle area DA3". Here, the refueling non-permission conditions for each of the confirmation-required areas DA1 to DA3 will be described.

The refueling non-permission conditions for the vehicle area DA1 include, for example, that the vehicle C is not parked in the refueling area P, that the door of the vehicle C is open, or that the window of the vehicle C is open. If the vehicle C is not parked in the refueling area P, that the door of the vehicle C is open, or that the window of the vehicle C is open, the first judgment result indicates that the vehicle area DA1 is an "NG area." In other words, the first judgment result indicates that a situation has occurred in which refueling cannot be permitted in the vehicle area DA1. Here, the fact that the vehicle C is not parked in the refueling area P includes, in addition to the fact that the vehicle C is not parked in the refueling area P, that there is a refueling target other than the vehicle C in the refueling area P. The refueling target other than the vehicle C is, for example, a portable can or a plastic container.

The conditions for not allowing refueling in the person area DA2 include, for example, that person A present in the person area DA2 is engaging in inappropriate behavior. If person A is engaging in inappropriate behavior, the first judgment result indicates that the person area DA2 is an "NG area." In other words, the first judgment result indicates that a situation has occurred in the person area DA2 in which refueling cannot be permitted. Here, inappropriate behavior includes person A being a child, person A smoking a cigarette, person A refueling a refueling target other than vehicle C, and the presence of multiple people in the person area DA2.

The refueling non-permission condition for the nozzle area DA3 includes, for example, that the refueling nozzle 41 is not inserted into the fuel filler opening of the vehicle C. When the refueling nozzle 41 is not inserted into the fuel filler opening of the vehicle C, the first judgment result indicates that the nozzle area DA3 is an "NG area." In other words, the first judgment result indicates that a situation has occurred in which refueling action is not permitted in the nozzle area DA3. Here, the refueling nozzle 41 not being inserted into the fuel filler opening of the vehicle C includes, in addition to the refueling nozzle 41 not being inserted into the fuel filler opening of the vehicle C, the refueling nozzle 41 being inserted into a refueling target other than the vehicle C, etc.

Next, the configuration of the mobile terminal 24 will be described with reference to FIG. 6. FIG. 6 is a block diagram showing the configuration of the mobile terminal 24 of this embodiment. As shown in FIG. 6, the mobile terminal 24 includes a device communication unit 241, a device processing unit 242, a device storage unit 243, and a touch display 244.

The terminal communication unit 241 controls communication between the video processing server 22 and the SSC 6 described with reference to FIG. 1. The terminal communication unit 241 has, for example, a wireless LAN board or a wireless LAN module.

The device communication unit 241 is an example of a display image receiving unit, a determination result information receiving unit, an operation information transmitting unit, and a request signal transmitting unit. The device communication unit 241 receives the display image (captured image T and first presentation image PR1) described with reference to FIG. 1 from the video processing server 22. The device communication unit 241 receives the determination result information described with reference to FIG. 4 from the video processing server 22. The device communication unit 241 transmits the operation information described with reference to FIG. 1 to the video processing server 22. The device communication unit 241 transmits the request signal described with reference to FIG. 1 to the video processing server 22.

The device storage unit 243 includes semiconductor memory such as ROM, RAM, and flash memory. The device storage unit 243 stores various computer programs executed by the device processing unit 242. The device storage unit 243 also stores various types of data.

The touch display 244 has a display device 2441 and a touch panel 2442. The display device 2441 is an example of a display unit. The touch panel 2442 is an example of a first operation receiving unit and a second operation receiving unit.

The display device 2441 displays the display image (captured image T and first presentation image PR1) described with reference to FIG. 1. The display device 2441 is, for example, a liquid crystal display or an organic EL (electroluminescence) display. The touch panel 2442 is superimposed on the display surface of the display device 2441. The touch panel 2442 accepts touch operations by an employee (operator). The touch panel 2442 accepts the refueling permission operation described with reference to FIG. 1.

The device processing unit 242 executes various processes. The device processing unit 242 has a processor such as a CPU or MPU. The processor executes various processes by executing computer programs stored in the device storage unit 243.

For example, the device processing unit 242 controls the device communication unit 241 to transmit the request signal (a signal requesting a display image from the video processing server 22) described with reference to FIG. 1 to the video processing server 22.

The device processing unit 242 also executes a decision process to determine whether or not to enable a refueling permission operation to permit a customer (refueling worker) to perform a refueling action. In detail, the device processing unit 242 decides whether or not to enable a refueling permission operation based on the judgment result information described with reference to FIG. 4. The device processing unit 242 is an example of a decision unit.

In this embodiment, the device processing unit 242 further generates a second presentation image PR2 based on the judgment result information. The second presentation image PR2 shows the judgment result for each area to be confirmed. Each judgment result indicates whether or not a situation has occurred in which refueling action is not permitted in the corresponding area to be confirmed. In other words, the second presentation image PR2 indicates whether or not the area to be confirmed includes an NG area. The second presentation image PR2 includes, for example, a text image or a mark image indicating the judgment result.

In this embodiment, the second presentation image PR2 includes a text image. Hereinafter, a text image indicating that the area to be confirmed is an NG image may be referred to as a "first text image." Similarly, a text image indicating that the area to be confirmed is an OK area may be referred to as a "second text image."

Next, the monitoring system 2 of this embodiment will be described with reference to Figs. 1 to 9. Figs. 7 to 9 are diagrams showing screens displayed by the mobile terminal 24 of this embodiment. In detail, Fig. 7 shows the mobile terminal 24 displaying the first screen G1. The first screen G1 is a refueling permission issuance screen. Hereinafter, the first screen G1 may be referred to as the "refueling permission issuance screen G1."

As shown in FIG. 7, the device processing unit 242 displays a refueling permission issuance screen G1 on the touch display 244. The refueling permission issuance screen G1 displays lane identification information LC, a captured image T, a first presentation image PR1, a second presentation image PR2, and a first button image B1.

In detail, in response to receiving the monitoring start signal described with reference to FIG. 1, the device processing unit 242 controls the device communication unit 241 to send a request signal (a signal requesting a display image) to the video processing server 22. As a result, the device processing unit 242 acquires the judgment result information together with the display image (captured image T and first presentation image PR1) via the device communication unit 241. The device processing unit 242 generates a second presentation image PR2 based on the judgment result information, and displays the second presentation image PR2 on the touch display 244 together with the display image (captured image T and first presentation image PR1).

The lane identification information LC is information for identifying the target lane LT (the refueling lane L to be monitored). In this embodiment, the lane identification information LC includes a text image indicating the target lane LT. In detail, as described with reference to FIG. 4, the request signal indicates the identification information of the target lane LT. The processing unit 222 of the video processing server 22 transmits the lane identification information LC (text image) to the mobile terminal 24 together with the display image and the judgment result information based on the identification information of the target lane LT.

The first button image B1 is a button image for authorizing the act of refueling. When an employee touches the first button image B1, the device processing unit 242 controls the device communication unit 241 to transmit a refueling permission signal to the SSC 6. The touch operation is, for example, a single tap. The operation of touching the first button image B1 is an example of a refueling permission operation. Hereinafter, the first button image B1 may be referred to as the "refueling permission button B1."

In this embodiment, as shown in FIG. 7, the first presentation image PR1 includes a text image "Car Area" indicating the vehicle area DA1, a text image "Person Area" indicating the person area DA2, and a text image "Nozzle" indicating the nozzle area DA3. The second presentation image PR2 includes a text image "NG" as the first text image (a text image indicating that the area to be confirmed is an NG image) and a text image "OK" as the second text image (a text image indicating that the area to be confirmed is an OK image).

In the example shown in FIG. 7, the judgment results (first judgment results) for the vehicle area DA1 and the person area DA2 are "OK," meaning that refueling is permitted. On the other hand, the first judgment result for the nozzle area DA3 is "NG," meaning that refueling is not permitted.

When the judgment result information indicates that all of the areas to be confirmed DA1 to DA3 are OK areas, the device processing unit 242 decides to enable the refueling permission operation. On the other hand, when the judgment result information indicates that any of the areas to be confirmed DA1 to DA3 are NG areas, the device processing unit 242 decides not to enable the refueling permission operation. In other words, the device processing unit 242 decides to disable the refueling permission operation.

In the example shown in FIG. 7, the judgment result (first judgment result) for the nozzle area DA3 is "NG," so the device processing unit 242 decides to disable the refueling permission operation. In this embodiment, when the device processing unit 242 decides to disable the refueling permission operation, it grays out the refueling permission button B1 as shown in FIG. 7, thereby disabling touch operations on the refueling permission button B1.

Figure 8 shows a mobile terminal 24 displaying a refueling permission issuance screen G1. More specifically, the refueling permission issuance screen G1 in Figure 8 displays the captured image T obtained after the captured image T shown in Figure 7 was captured.

As shown in FIG. 8, if the determination result information indicates that all of the check-required areas DA1 to DA3 are OK areas, the device processing unit 242 determines to enable the refueling permission operation. Specifically, as shown in FIG. 8, the device processing unit 242 enables the touch operation on the refueling permission button B1.

Figure 9 shows an example of a refueling permission issuance screen G1 that is displayed on the mobile terminal 24 when the first judgment result by the video processing server 22 includes an erroneous judgment. In the example shown in Figure 9, the judgment result (first judgment result) of the vehicle area DA1 and the person area DA2 is "OK", and the judgment result (first judgment result) of the nozzle area DA3 is "NG". In this case, the employee checks the nozzle area DA3 via the refueling permission issuance screen G1, and if there is no factor in the nozzle area DA3 that would disallow the refueling action, performs a touch operation on the nozzle area DA3.

When an employee performs a touch operation, the device processing unit 242 controls the device communication unit 241 to transmit, based on a signal output from the touch panel 2442, coordinate information (touch coordinates) indicating the position where the touch operation was performed in the captured image T and touch operation information indicating the type of touch operation to the video processing server 22. The coordinate information and touch operation information are examples of operation information (information indicating the content of the operation performed on the displayed image).

When the processing unit 222 of the video processing server 22 acquires the coordinate information and the touch operation information via the communication unit 221, the processing unit 222 determines whether or not a predetermined operation has been performed on the confirmation-required areas DA1 to DA3 in the captured image T based on the coordinate information and the touch operation information (second determination process). The predetermined operation is, for example, a single tap.

The processing unit 222 of the video processing server 22 executes a fixing process based on the second judgment result (the judgment result of the second judgment process). Specifically, when the processing unit 222 of the video processing server 22 judges that a predetermined operation has been performed on the confirmation-required areas DA1 to DA3 in the captured image T, it fixes the judgment result of the confirmation-required area where the predetermined operation has been performed to "OK". Then, it controls the communication unit 221 to transmit the judgment result information after the fixing process to the mobile terminal 24.

For example, if an employee single-tap on nozzle area DA3, the second judgment result indicates that nozzle area DA3 has been confirmed. Therefore, the processing unit 222 of the video processing server 22 fixes the judgment result for nozzle area DA3 to "OK." Therefore, the judgment result information indicates that nozzle area DA3 is an OK area.

When the device processing unit 242 receives the determination result information after the fixing process via the device communication unit 241, it updates the second presentation image PR2 based on the determination result information after the fixing process. For example, if the nozzle area DA3 is an OK area in the determination result information after the fixing process, the device processing unit 242 changes (fixes) the character image for the nozzle area DA3 in the second presentation image PR2 from the first character image (NG) to the second character image (OK).

When the device processing unit 242 receives the determination result information after the fixing process via the device communication unit 241, it determines whether or not to enable the refueling permission operation based on the determination result information after the fixing process. For example, in the example shown in FIG. 9, if the determination result for the nozzle area DA3 is fixed to "OK", all of the areas requiring confirmation DA1 to DA3 become OK areas, so it determines to enable the refueling permission operation. As a result, as shown in FIG. 8, the touch operation on the refueling permission button B1 becomes enabled.

Next, the processing executed by the processing unit 222 of the video processing server 22 will be described with reference to Figures 1 to 10. Figure 10 is a flowchart showing the processing executed by the processing unit 222 of the video processing server 22 of this embodiment. The processing shown in Figure 10 is started when the mobile terminal 24 transmits a request signal to the video processing server 22. Note that the video processing server 22 constantly acquires the first captured image T1 and the second captured image T2 from the first imaging device 26a and the second imaging device 26b, and constantly extracts the confirmation-required area in the first captured image T1 and the confirmation-required area in the second captured image T2.

As shown in FIG. 10, when the processing unit 222 of the video processing server 22 receives a request signal from the mobile terminal 24, the processing unit 222 executes a first determination process for a confirmation-required area in the captured image T of the target lane LT (step S1). As described with reference to FIG. 4, the first determination process is a process for determining whether or not the image of the confirmation-required area is an image that matches the refueling non-permission condition.

The processing unit 222 of the video processing server 22 executes the first judgment process to generate judgment result information. The judgment result information indicates whether the judgment result for each confirmation-required area is "NG" or not.

After executing the first judgment process, the processing unit 222 of the video processing server 22 controls the communication unit 221 to transmit judgment result information to the mobile terminal 24 together with the captured image T of the target lane LT and the first presented image PR1 (display image) (step S2).

After transmitting the display image and the judgment result information, the processing unit 222 of the video processing server 22 judges whether the area to be confirmed includes an NG area based on the judgment result information (step S3).

If the processing unit 222 of the video processing server 22 determines that the area to be confirmed does not include an NG area (No in step S3), the process shown in FIG. 10 ends.

When the processing unit 222 of the video processing server 22 determines that the area to be confirmed is included in the NG area (Yes in step S3), it determines whether or not operation information (coordinate information and touch operation information) has been received from the mobile terminal 24 (step S4).

If the processing unit 222 of the video processing server 22 determines that no operation information has been received from the mobile device 24 (No in step S4), the process returns to step S1.

When the processing unit 222 of the video processing server 22 determines that operation information has been received from the mobile terminal 24 (Yes in step S4), the processing unit 222 determines whether or not a predetermined operation has been performed on the confirmation-required area in the captured image T based on the operation information (step S5). In other words, the processing unit 222 of the video processing server 22 executes a second determination process.

If the processing unit 222 of the video processing server 22 determines that a predetermined operation has not been performed on the area requiring confirmation in the captured image T (No in step S5), the process returns to step S1.

When the processing unit 222 of the video processing server 22 determines that a predetermined operation has been performed on the confirmation-required area in the captured image T (Yes in step S5), it executes a fixing process (step S6). In detail, the processing unit 222 of the video processing server 22 fixes the determination result for the confirmation-required area where the predetermined operation has been performed to "OK" in the determination result information.

After the fixing process is performed, the process returns to step S1. As already explained, the video processing server 22 acquires the first captured image T1 and the second captured image T2 from the first imaging device 26a and the second imaging device 26b at any time, and extracts the confirmation-required area in the first captured image T1 and the confirmation-required area in the second captured image T2 at any time. Therefore, when returning to step S1 from step S4, step S5, or step S6, the determination result of the first determination process may change. Note that, since the determination result of the confirmation-required area in which a predetermined operation has been performed is fixed to "OK" by the fixing process, the processing unit 222 of the video processing server 22 performs the first determination process on the confirmation-required areas whose determination result is not fixed to "OK" among the confirmation-required areas.

Next, the processing executed by the device processing unit 242 will be described with reference to Figs. 1 to 11. Fig. 11 is a flowchart showing the processing executed by the device processing unit 242 of the mobile device 24 in this embodiment. The processing shown in Fig. 11 is started when the mobile device 24 transmits a request signal to the video processing server 22.

As shown in FIG. 11, the device processing unit 242 transmits a request signal to the video processing server 22, and then receives the judgment result information and the display image of the target lane LT (captured image T and first presentation image PR1) from the video processing server 22 (step S11).

When the device processing unit 242 receives the determination result information and the display image of the target lane LT from the video processing server 22, it generates the second presentation image PR2 described with reference to FIG. 6 based on the determination result information (step S12). Then, the device processing unit 242 displays the second presentation image PR2 on the touch display 244 together with the display image (captured image T and first presentation image PR1).

After displaying the display image (captured image T and first presentation image PR1) and the second presentation image PR2 on the touch display 244, the device processing unit 242 determines whether or not to enable the refueling permission operation based on the judgment result information (step S13). More specifically, the device processing unit 242 determines whether or not the confirmation-required area includes an NG area based on the judgment result information.

When the device processing unit 242 determines that the confirmation-required area includes an NG area, it decides not to enable the refueling permission operation (No in step S13). In other words, the device processing unit 242 decides to disable the refueling permission operation.

After deciding not to enable the refueling permission operation, the device processing unit 242 determines whether or not a touch operation has been performed based on the signal output from the touch display 244 (touch panel 2442) (step S14).

If the device processing unit 242 determines that a touch operation has not been performed (No in step S14), the process returns to step S11.

If the device processing unit 242 determines that a touch operation has been performed (Yes in step S14), it controls the device communication unit 241 to transmit operation information (coordinate information and touch operation information) to the video processing server 22 (step S15). As a result, the process returns to step S11.

If the device processing unit 242 determines that the confirmation-required area does not include an NG area, it decides to enable the refueling permission operation (Yes in step S13). After deciding to enable the refueling permission operation, the device processing unit 242 determines whether the refueling permission button B1 described with reference to FIG. 7 has been touched (step S16).

The device processing unit 242 waits until the refueling permission button B1 is touched (No in step S16). If the device processing unit 242 determines that the refueling permission button B1 has been touched (Yes in step S16), it controls the device communication unit 241 to transmit a refueling permission signal to the SSC 6 (step S17), and ends the processing shown in FIG. 11.

As described with reference to FIG. 10, the determination result information generated by the video processing server 22 and the captured image T acquired by the video processing server 22 from the imaging device 26 are updated as needed. Therefore, the determination result information and the display image of the target lane LT received by the device processing unit 242 in step S11 are updated as needed.

The first embodiment of the present invention has been described above with reference to Figs. 1 to 11. According to this embodiment, the mobile terminal 24 displays the first presentation image PR1 (an image presenting an area requiring confirmation within the captured image T), so that customers at a refueling station can more reliably confirm the safety of their own refueling actions.

In addition, according to this embodiment, the mobile terminal 24 displays the second presentation image PR2 (an image indicating whether the confirmation area is an NG area or not), so that customers at the refueling station can more reliably confirm the safety of their own refueling actions.

In addition, according to this embodiment, the refueling permission operation is not valid until an employee checks the confirmation-required area that the video processing server 22 has determined to be an NG area. Therefore, it is possible to more reliably check the safety of refueling actions by customers at the refueling station themselves.

In this embodiment, the device processing unit 242 generates the second presentation image PR2, but the processing unit 222 of the video processing server 22 may generate the second presentation image PR2.

In addition, in this embodiment, the video processing server 22 transmits information indicating the judgment result for each confirmation-required area as judgment result information to the mobile terminal 24, but the video processing server 22 may transmit the second presentation image PR2 as judgment result information to the mobile terminal 24.

In this case, the processing unit 222 of the video processing server 22 generates a second presentation image PR2 based on the first judgment result and the second judgment result. Also, the device processing unit 242 does not generate the second presentation image PR2, but acquires a judgment result for each confirmation-required area based on the second presentation image PR2 received from the video processing server 22. For example, the device processing unit 242 may acquire the judgment result by character recognition processing. The character recognition processing is, for example, optical character recognition (OCR) processing.

[Embodiment 2]
Next, a second embodiment of the present invention will be described with reference to Figures 1 to 6 and 12 to 14. However, differences from the first embodiment will be described, and descriptions of the same matters as in the first embodiment will be omitted. In the second embodiment, unlike the first embodiment, the video processing server 22 does not execute the first determination process.

Figure 12 is a diagram showing a screen displayed by the mobile device 24 of this embodiment. In detail, Figure 12 shows the mobile device 24 displaying the refueling permission issuance screen G1. As shown in Figure 12, the mobile device 24 of this embodiment does not display the second presentation image PR2 described with reference to Figure 6. In this embodiment, the refueling permission operation is enabled when a predetermined operation is performed on the entire area of the confirmation required area.

Next, the processing executed by the processing unit 222 of the video processing server 22 will be described with reference to Figures 1 to 6, 12, and 13. Figure 13 is a flowchart showing the processing executed by the processing unit 222 of the video processing server 22 of this embodiment. The processing shown in Figure 13 is started when the mobile terminal 24 transmits a request signal to the video processing server 22. Note that, similar to the first embodiment, the video processing server 22 constantly acquires the first captured image T1 and the second captured image T2 from the first imaging device 26a and the second imaging device 26b, and constantly extracts the confirmation-required area in the first captured image T1 and the confirmation-required area in the second captured image T2.

As shown in FIG. 13, when the processing unit 222 of the video processing server 22 receives a request signal from the mobile terminal 24, it controls the communication unit 221 to transmit a display image (the first presentation image PR1 and the captured image T of the target lane LT) to the mobile terminal 24 (step S21).

After transmitting the display image (the first presentation image PR1 and the captured image T of the target lane LT), the processing unit 222 of the video processing server 22 determines whether or not operation information (coordinate information and touch operation information) has been received from the mobile terminal 24 (step S22).

If the processing unit 222 of the video processing server 22 determines that no operation information has been received from the mobile device 24 (No in step S22), the process returns to step S21.

When the processing unit 222 of the video processing server 22 determines that operation information has been received from the mobile terminal 24 (Yes in step S22), the processing unit 222 determines whether or not a predetermined operation has been performed on the confirmation-required area in the captured image T based on the operation information (step S23). In other words, the processing unit 222 of the video processing server 22 executes a second determination process.

If the processing unit 222 of the video processing server 22 determines that a predetermined operation has not been performed on the area requiring confirmation in the captured image T (No in step S23), the process returns to step S21.

When the processing unit 222 of the video processing server 22 determines that a predetermined operation has been performed on the area to be confirmed in the captured image T (Yes in step S23), it controls the communication unit 221 to transmit the determination result information to the mobile terminal 24 (step S24). Here, the determination result information indicates the second determination result. In other words, the determination result information is information indicating whether or not the area to be confirmed has been confirmed.

After transmitting the determination result information, the processing unit 222 of the video processing server 22 determines whether or not the entire area of the area to be confirmed has been confirmed based on the determination result information (step S25). In other words, the processing unit 222 of the video processing server 22 determines whether or not a predetermined operation has been performed on the entire area of the area to be confirmed based on the determination result information.

If the processing unit 222 of the video processing server 22 determines that any of the areas to be confirmed have not been confirmed (No in step S25), the process returns to step S21.

If the processing unit 222 of the video processing server 22 determines that the entire area to be confirmed has been confirmed (Yes in step S25), it ends the processing shown in FIG. 13.

Next, the process executed by the device processing unit 242 of the mobile terminal 24 will be described with reference to Figures 1 to 6 and Figures 12 to 14. Figure 14 is a flowchart showing the process executed by the device processing unit 242 of the mobile terminal 24 in this embodiment. The process shown in Figure 14 is started when the mobile terminal 24 transmits a request signal to the video processing server 22.

As shown in FIG. 14, after transmitting a request signal to the video processing server 22, the device processing unit 242 receives a display image (captured image T and first presentation image PR1) of the target lane LT from the video processing server 22 (step S31). When the device processing unit 242 receives the display image of the target lane LT from the video processing server 22, it causes the display image to be displayed on the touch display 244.

After displaying the display image on the touch display 244, the device processing unit 242 determines whether or not a touch operation has been performed based on the signal output from the touch display 244 (touch panel 2442) (step S32).

If the device processing unit 242 determines that a touch operation has not been performed (No in step S32), the process returns to step S31.

If the device processing unit 242 determines that a touch operation has been performed (Yes in step S32), it controls the device communication unit 241 to transmit operation information (coordinate information and touch operation information) to the video processing server 22 (step S33).

After transmitting the operation information, the device processing unit 242 determines whether or not it has received determination result information from the video processing server 22 (step S34). Here, the determination result information indicates the second determination result. In other words, the determination result information is information indicating whether or not a confirmation-required area has been confirmed.

If the device processing unit 242 determines that no judgment result information has been received from the video processing server 22 (No in step S34), the process returns to step S31.

When the device processing unit 242 determines that it has received the judgment result information from the video processing server 22 (Yes in step S34), it determines whether or not to enable the refueling permission operation based on the judgment result information (step S35). More specifically, the device processing unit 242 determines whether or not the entire area of the area requiring confirmation has been confirmed based on the judgment result information.

When the device processing unit 242 determines that some areas of the area requiring confirmation have not been confirmed, it decides not to enable the refueling permission operation (No in step S35). In other words, the device processing unit 242 decides to disable the refueling permission operation. As a result, the process returns to the process in step S31.

If the device processing unit 242 determines that the entire confirmation area has been confirmed, it decides to enable the refueling permission operation (Yes in step S35). After deciding to enable the refueling permission operation, the device processing unit 242 determines whether the refueling permission button B1 described with reference to FIG. 7 has been touched (step S36).

The device processing unit 242 waits until the refueling permission button B1 is touched (No in step S36). If the device processing unit 242 determines that the refueling permission button B1 is touched (Yes in step S36), it controls the device communication unit 241 to transmit a refueling permission signal to the SSC 6 (step S37), and ends the processing shown in FIG. 14.

The second embodiment of the present invention has been described above with reference to Figures 1 to 6 and Figures 12 to 14. According to this embodiment, similar to the first embodiment, the mobile terminal 24 displays the first presentation image PR1 (an image presenting an area requiring confirmation within the captured image T), so that customers at a refueling station can more reliably confirm the safety of their own refueling actions.

In addition, according to this embodiment, the refueling permission operation is not valid until an employee has checked the entire area of the area requiring checking. Therefore, it is possible to more reliably check the safety of refueling actions by customers at the refueling station themselves.

In this embodiment, the mobile terminal 24 does not display the second presentation image PR2, but the mobile terminal 24 may display the second presentation image PR2 indicating the second judgment result. In this case, the second presentation image PR2 includes only the second character image "OK" out of the first character image "NG" and the second character image "OK". In other words, the second presentation image PR2 presents information indicating the confirmation-required area where a predetermined touch operation has been performed.

The above describes embodiments of the present invention with reference to the drawings (Figs. 1 to 14). However, the present invention is not limited to the above embodiments, and can be implemented in various aspects without departing from the gist of the present invention. In addition, the components disclosed in the above embodiments can be modified as appropriate. For example, some of the components shown in one embodiment may be added to the components of another embodiment, or some of the components shown in one embodiment may be deleted from the embodiment.

The drawings are primarily schematic illustrations of each component to facilitate understanding of the invention, and the thickness, length, number, spacing, etc., of each component shown may differ from the actual configuration due to the convenience of creating the drawings. Furthermore, the configuration of each component shown in the above embodiment is merely an example and is not particularly limited, and it goes without saying that various modifications are possible within a range that does not substantially deviate from the effects of the present invention.

For example, in the embodiment described with reference to Figures 1 to 14, the weighing machine 4 transmits a fuel supply permission request signal to the SSC 6, but the weighing machine 4 does not have to transmit a fuel supply permission request signal to the SSC 6. In this case, the SSC 6 transmits a monitoring start signal to the mobile terminal 24 in response to receiving data indicating the type of fuel selected by the customer, the payment method, and the amount of fuel supplied or the payment amount.

In the embodiment described with reference to Figures 1 to 14, one imaging device 26 is provided for each refueling lane L or refueling area P, but multiple imaging devices 26 may be provided for each refueling lane L or refueling area P.

In addition, in the embodiment described with reference to Figures 1 to 14, if the device processing unit 242 decides to disable the refueling permission operation, the first button image B1 (refueling permission button B1) is grayed out, but if the device processing unit 242 decides to disable the refueling permission operation, the first button image B1 (refueling permission button B1) does not have to be displayed.

In the embodiment described with reference to Figures 1 to 14, the mobile terminal 24 is the refueling permission issuing terminal, but the refueling permission issuing terminal may be the SSC 6. In this case, the SSC 6 is equipped with at least one of a touch panel and a pointing device.

Furthermore, the mobile terminal 24 may enlarge and display the area of the captured image T where the touch operation was performed in response to a touch operation on the first screen G1 (fueling permission issuance screen). Below, a mobile terminal 24 of another embodiment will be described with reference to FIG. 15. FIG. 15 is a diagram showing a screen displayed by the mobile terminal 24 of another embodiment. In detail, FIG. 15 shows the mobile terminal 24 displaying the second screen G2. The second screen G2 displays lane identification information LC, an enlarged image TA, and a second button image B2.

When the device processing unit 242 detects, based on a signal output from the touch display 244 (touch panel 2442), that a touch operation has been performed to instruct the enlargement of a portion of the captured image T, the device processing unit 242 controls the device communication unit 241 to transmit operation information to the video processing server 22. Here, the operation information includes coordinate information and touch operation information. The touch operation information indicates the type of touch operation. For example, the touch operation that instructs the enlargement of a portion of the captured image T is a single tap. In this case, the touch operation indicating that the employee has confirmed the area requiring confirmation is a touch operation other than a single tap. For example, the touch operation indicating that the employee has confirmed the area requiring confirmation may be a double tap.

When the touch operation information included in the operation information received from the mobile device 24 indicates a touch operation instructing to enlarge a partial area of the captured image T, the processing unit 222 of the video processing server 22 generates an enlarged image TA in which a partial area of the captured image T is enlarged based on the coordinate information. Then, the processing unit 222 of the video processing server 22 controls the communication unit 221 to transmit the enlarged image TA to the mobile device 24. As a result, the device processing unit 242 causes the second screen G2 to be displayed on the touch display 244.

The second button image B2 is a button image for transitioning the screen displayed on the touch display 244 from the second screen G2 to the first screen G1. When an employee touches the second button image B2, the screen displayed on the touch display 244 transitions from the second screen G2 to the first screen G1 (the refueling permission issuance screen).

The present invention can be applied to self-service fuel stations and has industrial applicability.

2: Surveillance system 22: Video processing server 24: Portable terminal 26: Imaging device 221: Communication unit 222: Processing unit 223: Storage unit 241: Terminal communication unit 242: Terminal processing unit 243: Terminal storage unit 244: Touch display 2441: Display device 2442: Touch panel DA1 to DA3: Check required area PR1: First presentation image PR2: Second presentation image T: Captured image

Claims (10)

A monitoring device communicably connected to a refueling permission issuing terminal that accepts an operation for permitting a refueling operation by a refueling worker,
an image receiving unit that receives an image from an imaging device that captures an image of the periphery of a metering machine installed at a fuel station;
an area extraction unit that extracts from the captured image a plurality of confirmation-required areas that need to be confirmed in order to permit the fuel supply operation by the fuel supply worker;
an image generating unit that generates a display image including the captured image and a presentation image presenting the confirmation-required area;
a request signal receiving unit that receives a request signal requesting the display image output by the fuel supply permission issuing terminal based on the receipt of a monitoring start signal from a console device connected to the meter;
A display image transmission unit that transmits the display image to the refueling permission issuing terminal ;
a judgment result information generating unit that executes a first judgment process for judging, for each of the confirmation required areas, whether or not the image of the confirmation required area is an image that matches a refueling non-permission condition that prohibits the refueling action by the refueling worker based on the captured image, and generates judgment result information that indicates, for each of the confirmation required areas, whether or not the refueling action is permitted;
a determination result information transmitting unit that transmits the determination result information to the refueling permission issuing terminal;
an operation information receiving unit that receives operation information indicating the content of an operation performed on the display image from the refueling permission issuing terminal;
Equipped with
the monitoring device is not connected to the console device except via the refueling permission issuing terminal ,
the display image is an image that displays the plurality of confirmation-required regions in a identifiable manner on one of the captured images,
The determination result information generating unit
A monitoring device that generates the judgment result information that validates the refueling permission operation by the refueling permission issuing terminal when it determines, based on the operation information, that a specified operation has been performed for all of the NG areas even if the multiple areas requiring confirmation include an NG area in which the first judgment process has determined that the refueling action is not permitted . The determination result information generating unit
A second determination process is further performed to determine whether or not a predetermined operation has been performed on the confirmation-required region based on the operation information.
A monitoring device as described in claim 1, wherein when it is determined that the specified operation has been performed in the confirmation-required area, the judgment result for the confirmation-required area in which the specified operation has been performed is fixed to a judgment result indicating that the refueling action may be permitted. The determination result information generating unit
The monitoring device of claim 1, further comprising: a determining unit that determines whether a predetermined operation has been performed on the area to be confirmed based on the operation information; and, if it determines that the predetermined operation has been performed on the area to be confirmed , generates the determination result information indicating that confirmation of the area to be confirmed has been performed. The monitoring device according to claim 1 , wherein the display image transmitting unit transmits the display image in response to the request signal being received by the request signal receiving unit. A monitoring device communicably connected to a refueling permission issuing terminal that accepts an operation for permitting a refueling operation by a refueling worker;
An imaging device that captures an image of the periphery of a metering machine installed at a fuel station,
The monitoring device includes:
a captured image receiving unit that receives a captured image from the imaging device;
an area extraction unit that extracts from the captured image a plurality of confirmation-required areas that need to be confirmed in order to permit the fuel supply operation by the fuel supply worker;
an image generating unit that generates a display image including the captured image and a presentation image presenting the confirmation-required area;
a request signal receiving unit that receives a request signal requesting the display image output by the fuel supply permission issuing terminal based on the receipt of a monitoring start signal from a console device connected to the meter;
A display image transmission unit that transmits the display image to the refueling permission issuing terminal ;
a judgment result information generating unit that executes a first judgment process for judging, for each of the confirmation required areas, whether or not the image of the confirmation required area is an image that matches a refueling non-permission condition that prohibits the refueling action by the refueling worker based on the captured image, and generates judgment result information that indicates, for each of the confirmation required areas, whether or not the refueling action is permitted;
a determination result information transmitting unit that transmits the determination result information to the refueling permission issuing terminal;
an operation information receiving unit that receives operation information indicating the content of an operation performed on the display image from the refueling permission issuing terminal;
Equipped with
the monitoring device is not connected to the console device except via the refueling permission issuing terminal ,
the display image is an image that displays the plurality of confirmation-required regions in a identifiable manner on one of the captured images,
The determination result information generating unit
A monitoring system that generates the judgment result information that validates the refueling permission operation by the refueling permission issuing terminal when it is determined based on the operation information that a specified operation has been performed for all of the NG areas, even if the multiple areas requiring confirmation include an NG area in which the first judgment process has determined that the refueling action is not permitted . The monitoring system of claim 5 further comprising the refueling permit issuing terminal. a fuel supply permission issuing terminal that accepts an operation for permitting a fuel supply operation by a fuel supply worker;
A monitoring device communicably connected to the refueling permission issuing terminal,
The monitoring device includes:
an image receiving unit that receives an image from an imaging device that captures an image of the periphery of a metering machine installed at a fuel station;
an area extraction unit that extracts from the captured image a plurality of confirmation-required areas that need to be confirmed in order to permit the fuel supply operation by the fuel supply worker;
an image generating unit that generates a display image including the captured image and a presentation image presenting the confirmation-required area;
a request signal receiving unit that receives a request signal requesting the display image output by the fuel supply permission issuing terminal based on the receipt of a monitoring start signal from a console device connected to the meter;
A display image transmission unit that transmits the display image to the refueling permission issuing terminal ;
a judgment result information generating unit that executes a first judgment process for judging, for each of the confirmation required areas, whether or not the image of the confirmation required area is an image that matches a refueling non-permission condition that prohibits the refueling action by the refueling worker based on the captured image, and generates judgment result information that indicates, for each of the confirmation required areas, whether or not the refueling action is permitted;
a determination result information transmitting unit that transmits the determination result information to the refueling permission issuing terminal;
an operation information receiving unit that receives operation information indicating the content of an operation performed on the display image from the refueling permission issuing terminal;
Equipped with
the monitoring device is not connected to the console device except via the refueling permission issuing terminal ,
the display image is an image that displays the plurality of confirmation-required regions in a identifiable manner on one of the captured images,
The judgment result information generating unit
A monitoring system that generates the judgment result information that validates the refueling permission operation by the refueling permission issuing terminal when it is determined based on the operation information that a specified operation has been performed for all of the NG areas, even if the multiple areas requiring confirmation include an NG area in which the first judgment process has determined that the refueling action is not permitted . a display image receiving unit that receives from a monitoring device a display image including an image acquired from an imaging device that captures an image of the periphery of a weighing machine installed at a fuel station and a presentation image that identifiably presents on one of the imaged images a plurality of check-required areas that need to be checked in order to permit a fuel supply operator to perform a fuel supply operation;
A display unit that displays the display image;
a determination result information receiving unit that receives, from the monitoring device, determination result information indicating whether or not the refueling action is permitted for each of the check-required areas;
a decision unit that decides whether or not to enable a refueling permission operation for permitting the refueling action based on the determination result information ;
a first operation receiving unit that receives the refueling permission operation by an operator ;
a second operation receiving unit that receives an operation performed on the display image;
a request signal transmission unit that transmits a request signal to the monitoring device requesting the display image when a monitoring start signal is received from a console device connected to the weighing machine;
a terminal communication unit that, when the refueling permission operation is received, transmits a refueling permission signal indicating permission of the refueling action to the console device without going through the monitoring device;
Equipped with
The determination unit is
A refueling permission issuing terminal that enables the refueling permission operation when a specified operation is performed on all of the NG areas based on the operation accepted by the second operation accepting unit, even if the display image indicates that the multiple areas requiring confirmation include an NG area where the monitoring device has determined that the refueling action is not permitted . an operation information transmission unit that transmits operation information indicating the content of an operation performed on the display image to the monitoring device;
The monitoring device determines whether or not a predetermined operation has been performed on the confirmation-required area in the captured image based on the operation information,
The refueling permission issuing terminal of claim 8, wherein when the monitoring device determines that the specified operation has been performed in the area requiring confirmation, the judgment result information receiving unit receives the judgment result information in which the judgment result for the area requiring confirmation is fixed to a judgment result indicating that the refueling action may be permitted. an operation information transmission unit that transmits operation information indicating the content of an operation performed on the display image to the monitoring device ;
the determination result information receiving unit receives, when a predetermined operation is performed on the confirmation-required area, the determination result information indicating that the confirmation-required area has been confirmed from the monitoring device ;
The monitoring device determines whether or not the predetermined operation has been performed on the confirmation-required area in the captured image based on the operation information;
The refueling permission issuing terminal according to claim 8 , wherein the decision unit decides whether or not to validate the refueling permission operation based on the judgment result information.

Images