JP7587336B2 - Live camera system, server, GUI information generation method and program - Google Patents

Description

The present invention relates to a live camera system that allows the most recent images captured by cameras installed at multiple locations to be displayed on a user's terminal device, and in particular to improvements to the graphical user interface (GUI) displayed on the terminal device.

In recent years, live camera systems have become widespread, in which multiple cameras are installed around roads and public facilities, and the most recent images captured by each camera are made available to an unspecified number of users via a communication network. Users can access the live camera system via the Internet using their own terminal devices, such as personal computers (PCs) or smartphones, to see images of the congestion situation on roads they plan to drive on, or the weather conditions around public facilities they plan to visit.

A live camera system is a system that is used irregularly by an unspecified number of users using their own terminal devices. In other words, the system needs to be designed with the assumption that it will be used by users who are not familiar with where to install the cameras or how to use the live camera system, and an intuitive, easy-to-understand GUI is required.

For example, in the live camera system disclosed in Patent Document 1, a user first uses a browser function provided in a terminal device to access the web server of the live camera system. The terminal device acquires GUI information such as a web page from the web server, and displays a GUI screen on the screen of the terminal device, which shows a map of the shooting area and an icon showing the installation location of the camera superimposed on the map.

When a user clicks on an icon on the map on the GUI screen, the terminal device displays a list of thumbnail images taken by each camera installed at the location indicated by the icon on the map on the terminal device screen based on the acquired GUI information.

When the user selects one of the thumbnail images displayed in the list, the terminal device retrieves the video image from the camera associated with the selected thumbnail image from the live camera system's server (referred to as a cloud server in Patent Document 1) and displays it.

Note that converting images captured by a camera successively into thumbnail images reduced to a desired size and having the camera periodically transmit the thumbnail images is not limited to live camera systems, but is a common practice. For example, by linking the thumbnail images received from the camera to GUI information, the user's terminal device that has acquired the GUI information can display the most recently captured thumbnail image on the GUI screen displayed based on the GUI information.

JP 2019-165319 A

However, as in the live camera system disclosed in Patent Document 1, even if thumbnail images are displayed on the GUI screen of a terminal device and a user selects one of the thumbnail images, there are cases in which the video image from the camera associated with the selected thumbnail image cannot be displayed on the user's terminal device.

One of the reasons why video images cannot be displayed may be a problem with the network between the camera and the server, or a malfunction of the camera itself.

Many users of live camera systems are unaware of how they work. Therefore, if they are unable to display video from a selected camera on their terminal device, they may file a complaint with the provider of the live camera system.

The present invention aims to provide a live camera system, server, GUI information generation method, and program that, when there is a possibility that the latest image from a specific camera cannot be transmitted from the server to the terminal device, automatically notifies the user before the user selects a camera that the image from the camera will not be displayed, thereby reducing inquiries from the user.

(1) In order to solve the above problems, a live camera system, which is one aspect of the present invention, includes a plurality of cameras that sequentially capture images, a server that receives video data generated based on the images from each camera via a network, and a terminal device that sequentially obtains streaming data converted from the video data from the server and is capable of displaying the images.

The server includes an acquisition unit, an update unit, a determination unit, a generation unit, and a server communication unit. The acquisition unit sequentially acquires thumbnail images obtained by reducing images captured by each camera. The thumbnail images include identification information that identifies the camera that captured the images. When the acquisition unit acquires the thumbnail images, the update unit updates the thumbnail images having the same identification information, and stores the thumbnail images in association with an update date and time indicating the date and time of the update. The determination unit determines at a predetermined timing whether a difference between the update date and time associated with the updated thumbnail image and the current date and time is within a predetermined time. When the determination unit determines that the difference is within the predetermined time, the generation unit generates GUI information including a thumbnail image and a reception component that receives a request to transmit streaming data of the camera identified by the identification information associated with the thumbnail image, and when the determination unit determines that the difference is not within the predetermined time, generates GUI information including non-display information indicating that the latest image of the camera identified by the identification information associated with the thumbnail image cannot be displayed instead of the thumbnail image. When a GUI request for GUI information is received from a terminal device via the network, the server communication unit transmits the GUI information to the terminal device.

The terminal device includes a terminal communication unit and a GUI unit. The terminal communication unit transmits a GUI request to the server and receives GUI information from the server. The GUI unit displays a GUI screen including thumbnail images and reception components or non-display information based on the GUI information, and accepts selection of the reception components.

(2) In one aspect of the present invention, in the live camera system of (1) above, the GUI information may include a map image and camera icons indicating the location where each camera is installed, and the GUI unit may display multiple camera icons superimposed on the map image based on the GUI information, and upon receiving a selection of one of the multiple camera icons, display a thumbnail image and a reception part of the camera installed at the location indicated by the camera icon, or non-display information.

(3) In one aspect of the present invention, in the live camera system of (1) above, the GUI information includes a map image and location icons indicating locations where multiple cameras are installed, and the GUI unit displays the multiple location icons superimposed on the map image based on the GUI information, and when a selection of one of the multiple location icons is accepted, a list including thumbnail images and reception parts or non-display information of each camera installed at the location indicated by the location icon is displayed based on the GUI information.

(4) In one aspect of the present invention, in the live camera system described in any one of (1) to (3) above, the thumbnail image may further include a shooting date and time indicating the date and time when the image was captured, and the determination unit may use the shooting date and time instead of the update date and time.

(5) In one aspect of the present invention, in the live camera system described in any one of (1) to (4) above, the server communication unit may further transmit the GUI information to the terminal device via the network when the generation unit generates the GUI information.

(6) In order to solve the above problems, a server according to one aspect of the present invention receives, via a network, video data generated based on images from a plurality of cameras that sequentially capture images, and when a request to transmit streaming data converted from the video data is received from a terminal device, the server sequentially transmits the streaming data to the terminal device via the network. The server includes an acquisition unit, an update unit, a determination unit, a generation unit, and a server communication unit. The acquisition unit sequentially acquires thumbnail images that are reduced versions of images captured by each camera. The thumbnail images include identification information that identifies the camera that captured the images. When the acquisition unit acquires the thumbnail images, the update unit updates the thumbnail images that have the same identification information, and stores the thumbnail images in association with an update date and time that indicates the date and time of the update. The determination unit determines, at a predetermined timing, whether the difference between the update date and time associated with the updated thumbnail image and the current date and time is within a predetermined time. If the determination unit determines that the difference is within a predetermined time, the generation unit generates GUI information including a thumbnail image and a reception component that receives a request to send streaming data from the camera identified by the identification information associated with the thumbnail image, and if the determination unit determines that the difference is not within the predetermined time, generates GUI information including non-display information indicating that the latest image captured by the camera identified by the identification information associated with the thumbnail image cannot be displayed, instead of the thumbnail image. If the server communication unit receives a GUI request requesting GUI information from the terminal device via the network, it transmits the GUI information to the terminal device.

(7) In order to solve the above problem, a GUI information generation method according to one aspect of the present invention is a GUI information generation method used in a server that receives, via a network, video image data generated based on images from a plurality of cameras that sequentially capture images, and that, upon receiving a request from a terminal device to transmit streaming data converted from the video image data, sequentially transmits the streaming data to the terminal device via the network. The GUI information generation method includes an acquisition step, an update step, a determination step, and a generation step. The acquisition step sequentially acquires thumbnail images that are reduced versions of images from each camera. The acquired thumbnail images include identification information that identifies the camera that captured the images. When a thumbnail image is acquired in the acquisition step, the update step updates a thumbnail image with the same identification information, and stores the thumbnail image in association with an update date and time indicating the date and time of the update. The determination step determines, at a predetermined time, whether the difference between the update date and time associated with the updated thumbnail image and the current date and time is within a predetermined time. In the generation step, if it is determined in the determination step that the difference is within a predetermined time, GUI information including a thumbnail image and a reception component that receives a request to send streaming data from the camera identified by the identification information associated with the thumbnail image is generated, and if it is determined that the difference is not within the predetermined time, GUI information including non-display information indicating that the latest image from the camera identified by the identification information associated with the thumbnail image cannot be displayed is generated instead of the thumbnail image.

(8) In order to solve the above problem, a program according to one aspect of the present invention is a program that receives, via a network, video data generated based on images from a plurality of cameras that sequentially capture images, and when a request to transmit streaming data converted from the video data is received from a terminal device, causes the program to function as a server that sequentially transmits the streaming data to the terminal device via the network, and causes the server to execute a process of generating GUI information. The generation process includes an acquisition step, an update step, a determination step, and a generation step. The acquisition step sequentially acquires thumbnail images that are reduced versions of images from each camera. The acquired thumbnail images include identification information that identifies the camera that captured the images. When a thumbnail image is acquired in the acquisition step, the update step updates a thumbnail image with the same identification information, and stores the thumbnail image in association with an update date and time indicating the date and time of the update. The determination step determines, at a predetermined time, whether the difference between the update date and time associated with the updated thumbnail image and the current date and time is within a predetermined time. In the generation step, if it is determined in the determination step that the difference is within a predetermined time, GUI information including a thumbnail image and a reception component that receives a request to send streaming data from the camera identified by the identification information associated with the thumbnail image is generated, and if it is determined that the difference is not within the predetermined time, GUI information including non-display information indicating that the latest image from the camera identified by the identification information associated with the thumbnail image cannot be displayed is generated instead of the thumbnail image.

When there is a possibility that the latest image from a particular camera cannot be sent from the server to the terminal device, the live camera system to which the present invention is applied automatically notifies the user before the user selects a camera that the latest image from that camera will not be displayed, thereby reducing inquiries from the user.

When a terminal device receives GUI information from a server to which this invention is applied, if there is a possibility that the server will not be able to send the latest images from a specific camera to the terminal device, the terminal device automatically notifies the user before the user selects a camera that the latest images from that camera will not be displayed, thereby reducing inquiries from the user.

When a terminal device receives GUI information from a server that executes a GUI information generation method or program to which the present invention is applied, if there is a possibility that the server cannot transmit the latest image of a specific camera to the terminal device, the terminal device automatically notifies the user before the user selects a camera that the latest image of the camera in question will not be displayed, thereby reducing inquiries from the user.

FIG. 1 is a diagram illustrating an example of the configuration of a live camera system 10. FIG. 2 is a diagram illustrating an example of a functional configuration of a terminal device 4. FIG. 2 illustrates an example of a functional configuration of a server 3. FIG. 2 is a sequence diagram of the live camera system 10. FIG. 2 is a diagram showing an example of a GUI screen based on GUI information of the present invention. FIG. 2 is a diagram showing an example of a GUI screen based on GUI information of the present invention. FIG. 1 is a diagram showing an example of a GUI screen based on conventional GUI information. FIG. 2 is a diagram showing an example of a GUI screen based on GUI information of the present invention. FIG. 13 is a diagram illustrating a modified example of the functional configuration of the server 3.

[composition]
A live camera system 10 according to an embodiment of the present invention will be described with reference to the drawings. As shown in Fig. 1, the live camera system 10 is composed of a camera 1, a network 2, a server 3, and a terminal device 4. The camera 1 is a general term for the first camera 1a or the second camera 1b. The network 2 is a general term for the first network 2a, the second network 2b, or the third network 2c. The terminal device 4 is a general term for the first terminal device 4a or the second terminal device 4b.

Camera 1 is a so-called network camera. Camera 1 has the function of sequentially capturing images and generating still image data or video image data based on the captured images. Camera 1 also has the function of generating thumbnail images, which are still image data in which the captured images have been reduced to a predetermined size, by setting the camera in advance.

Camera 1 has a communication function for communicating with other communication devices via network 2, and transmits the generated still image data, video data, or thumbnail images to a specified network device. In this embodiment, each camera 1 transmits the generated still image data, video data, or thumbnail images to server 3 via network 2.

Multiple cameras 1 are installed in a given area to sequentially capture road congestion and weather conditions near the installation location. Cameras 1 are installed, for example, on poles erected on the side of the road or at high points in facilities.

The first camera 1a and the second camera 1b are examples of the existence of multiple cameras 1 and the possibility of different communication formats, and are not limited to two cameras. The first camera 1a and the second camera 1b are installed so as to capture images of different shooting ranges. For example, the first camera 1a and the second camera 1b may be installed at different locations, or may be installed at the same location but with different shooting viewpoints and shooting directions.

The first camera 1a is connected to the first network 2a via a wired LAN cable. The second camera 1b is wirelessly connected to the second network 2b, which is a wireless LAN. The area in which the second camera 1b is installed has a poor wireless environment, so communication via the second network 2b sometimes fails. The second network 2b is connected to the first network 2a.

The server 3 has a function of receiving still image data, video data, or thumbnail images sequentially transmitted from the camera 1 via the network 2. When the server 3 receives a request from a terminal device 4, it has a function of transmitting, to the requesting terminal device 4, GUI information including the received thumbnail image or streaming data converted from the still image data or video data. The detailed configuration and operation of the server 3 will be described later.

The terminal device 4 is a PC or smartphone owned or used by a user. The terminal device 4 has a communication function and communicates with the server 3 via the network 2. The terminal device 4 receives GUI information and streaming data from the server 3 by sending a GUI request or a streaming request to the server 3. The terminal device 4 has a function of displaying a GUI screen based on the GUI information received from the server 3 via the network 2, and displaying an image captured by the selected camera 1 based on the received streaming data.

The first terminal device 4a and the second terminal device 4b are examples showing that there are multiple terminal devices 4 and that the communication format may differ depending on the terminal device 4. In other words, the number of terminal devices 4 is not limited to two. The first terminal device 4a is a so-called PC. The first terminal device 4a is connected to the first network 2a by a wired LAN cable.

The second terminal device 4b is a so-called smartphone. The second terminal device 4b has a wireless communication function for connecting to the third network 2c, which is a wireless communication network. The third network 2c is connected to the first network 2a. The detailed configuration and operation of the terminal device 4 will be described later.

[Configuration of Server 3]
2, the server 3 is composed of a web server 30, an FTP server 31, and a streaming server 32. The hardware of the server 3 is a computer equipped with a CPU, a memory, and a large-capacity storage medium. The server 3 may be a server group composed of a plurality of computers. The server 3 may also be a virtual machine constructed using the server group.

The web server 30 has a function of generating and transmitting GUI information, which is a web page written in HTML (HyperText Markup Language) or the like. When the web server 30 receives a request (hereinafter referred to as a "GUI request") specifying a URI (Uniform Resource Identifier) of the GUI information from the terminal device 4 via HTTP (HyperText Transfer Protocol) communication, the web server 30 transmits the specified GUI information to the terminal device 4 via HTTP communication.

The web server 30 includes a memory unit 33, a timing unit 34, an acquisition unit 35, an update unit 36, a determination unit 37, a generation unit 38, and a server communication unit 39. The memory unit 33 has a function of storing various information. The memory unit 33 stores, for example, various images constituting the GUI information, the latest thumbnail images captured by each camera 1, non-display information, reception components such as icons, programs related to the generation process of GUI information, etc.

The clock unit 34 has the function of clocking the current date and time. The hardware of the clock unit 34 is composed of an RTC (Real Time Clock) circuit.

The acquisition unit 35 has a function of sequentially acquiring thumbnail images that are reduced versions of images captured by each camera 1. Specifically, the acquisition unit 35 stores the thumbnail images received from each camera 1 by the FTP server 31 (described later) in a public folder provided in the storage unit 33.

The thumbnail image may be an image compressed in JPEG format, or may be in another image format, such as PNG, BMP, GIF, or another format. The thumbnail image contains identification information that identifies the camera 1 that took the image. If the thumbnail image is a JPEG format image, the Exif information contains the identification information and the date and time of shooting. The file format, image size, and transmission cycle of the thumbnail image are set for the camera 1.

When the acquisition unit 35 acquires a thumbnail image, the update unit 36 updates the thumbnail image with the same identification information that has already been stored in the public folder, and stores the associated update date and time indicating the date and time at which the update was performed. The update date and time is stored in a publicly known file system of the web server 30 using the current date and time measured by the clock unit 34. If a thumbnail image with the same identification information is not in the public folder, the thumbnail image is stored as is. Updating a thumbnail image means replacing thumbnail images with the same identification information that have older update dates and times with newer ones. Therefore, when a thumbnail image with a newer update date and time is acquired, the thumbnail image with the older update date and time is deleted from the storage unit 33.

The determination unit 37 has a function of determining, at a predetermined timing, whether the difference between the update date and time associated with the acquired thumbnail image and the current date and time is within a predetermined time. The predetermined time may be the same as or different from the period for transmitting thumbnail images from each camera 1 to the FTP server 31.

The specified time may be, for example, a cycle that is 1.5 times longer than the transmission cycle of thumbnail images. The cycle at which each camera 1 transmits thumbnail images may be the same, but the timing of transmission may be different. In addition, assuming that some network delay is tolerable, the time obtained by adding the network delay time to the transmission cycle of camera 1 may be set as the specified time for determination.

For example, since the second camera 1b is wirelessly connected to the second network 2b, it is possible that communication may be interrupted due to the influence of jamming radio waves or the like. However, communication may be restored after one minute. In this way, when there is a possibility that communication may be restored in a short period of time, an appropriate judgment can be made by adjusting the predetermined time used by the judgment unit 37 for judgment.

The generating unit 38 has a function of generating GUI information including the acquired thumbnail image, or GUI information including non-display information instead of the acquired thumbnail image. If the determining unit 37 determines that the difference between the update date and time of the acquired thumbnail image and the current time is within a predetermined time, the generating unit 38 generates GUI information including the thumbnail image and a reception component that receives a request to transmit streaming data of the camera 1 identified by the identification information associated with the thumbnail image.

On the other hand, if the determination unit 37 determines that the difference is not within the predetermined time, the generation unit 38 generates, instead of the thumbnail image, GUI information including non-display information indicating that the latest image of the camera 1 identified by the identification information associated with the thumbnail image cannot be displayed.

As described above, the server communication unit 39 has a communication function to transmit the GUI information generated by the generation unit 38 when it receives a GUI request from each terminal device 4 via HTTP communication.

The FTP server 31 has a function of sequentially receiving thumbnail images generated at a predetermined interval from each camera 1 via FTP communication. The thumbnail images sequentially received by the FTP server 31 are temporarily stored in an FTP folder provided in a storage unit (not shown) within the FTP server 31. The acquisition unit 35 monitors the FTP folder, and when a new thumbnail image is stored, it acquires it and stores it in a public folder in the storage unit 33. When the acquisition unit 35 acquires a thumbnail image from the FTP folder, it deletes the thumbnail image from the FTP folder.

The streaming server 32 has a function of sequentially receiving video data from each camera 1 via the network 2 and converting the received video data into streaming data of a predetermined streaming format, and a communication function of sequentially transmitting the specified streaming data to the terminal device 4 upon receiving a request to transmit streaming data specified from the terminal device 4 (hereinafter referred to as a "streaming request").

The streaming method used by the streaming server 32 is preferably, for example, the HTTP streaming method, and more specifically, it is preferably a method compatible with standards such as HLS (HTTP Live Streaming) and MPEG DASH. Note that a unique protocol other than the HTTP streaming method may also be used. The function of the streaming server 32 relating to conversion to streaming data and transmission is itself a publicly known technology, as described in, for example, JP 2020-136702 A, and therefore a detailed description is omitted.

[Configuration of terminal device 4]
3, the terminal device 4 is composed of a terminal communication unit 41, a terminal storage unit 42, a terminal control unit 43, and a GUI unit 44. As described above, the terminal device 4 is a computer equipped with a CPU, a memory, and a storage medium, and each functional unit is realized by the CPU executing a control program stored in the storage medium.

The terminal communication unit 41 has a communication function to send a GUI request or a streaming request to the server 3 via the network 2 and receive GUI information or streaming data from the server 3.

The terminal storage unit 42 has the above-mentioned storage medium, and stores the control program that controls the terminal device 4, as well as the received GUI information and streaming data. The terminal control unit 43 is composed of the above-mentioned CPU and memory, and controls each functional unit of the terminal device 4.

Although not shown in the figure, the hardware of the GUI unit 44 includes a display for displaying information and an input function for accepting selections from the user. For example, the hardware of the GUI unit 44 in the first terminal device 4a can include a display, a mouse, and a keyboard. Also, the GUI unit 44 in the second terminal device 4b, which is a smartphone, can include a touch panel display.

The GUI unit 44 has a function of displaying a GUI screen including thumbnail images and reception components or non-display information based on the GUI information received by the terminal communication unit 41, and accepting selection of the reception components. The GUI unit 44 also has a function of displaying an image captured by the camera 1 based on the received streaming data.

[Action]
The operation of the live camera system 10 will be described with reference to Fig. 4. Note that the symbol "S" used in the description of the operation means a step. First, as shown in Fig. 4, when the acquisition unit 35 acquires a thumbnail image (acquisition step, S0: Yes), the server 3 proceeds to processing of S1, and when the acquisition unit 35 does not acquire a thumbnail image (acquisition step, S0: No), the server 3 proceeds to processing of S3.

In the process of S1, if there is a thumbnail image associated with the same identification information as the identification information associated with the acquired thumbnail image (S1: Yes), the update unit 36 updates it to the newly acquired thumbnail image (update step, S2). In other words, the update unit 36 deletes the old thumbnail image and stores the newly acquired thumbnail image and the update date and time. Then, the process proceeds to S3.

In the process of S1, if there is no thumbnail image associated with the same identification information as the identification information associated with the acquired thumbnail image (S1: No), proceed to the process of S3.

In the process of S3, if the timing unit 34 determines that the predetermined time has elapsed (S3: Yes), the process proceeds to the judgment step S4, and if the time has not elapsed (S3: No), the process waits until the time has elapsed.

In the process of S4, the determination unit 37 determines whether the difference between the update date and time associated with the acquired thumbnail image and the current date and time is within a predetermined time. If the determination unit 37 determines that the difference is within the predetermined time (determination step, S4: Yes), the process proceeds to the generation step of S5. If the determination unit 37 determines that the difference is not within the predetermined time (determination step, S4: No), the process proceeds to the process of S6.

In S5 (generation step), the generation unit 38 generates GUI information including a thumbnail image and a reception component that receives a request to transmit streaming data of the camera 1 identified by the identification information associated with the thumbnail image.

In S6 (generation step), the generation unit 38 generates, in place of the thumbnail image, GUI information including non-display information indicating that the latest image of the camera 1 identified by the identification information associated with the thumbnail image cannot be displayed.

After the generation step of S5 or S6, when the GUI unit of the terminal device 4 accepts a GUI request (S7), the server communication unit 39 of the server 3 receives the GUI request. The server communication unit 39 transmits the GUI information generated by the generation unit 38 to the terminal device 4 (S8).

The GUI unit 44 of the terminal device 4 displays a GUI screen including a thumbnail image or non-display information based on the received GUI information (S9).

The GUI screen displayed by the GUI unit 44 of the terminal device 4 will be described with reference to the drawings. FIG. 5 is a diagram showing an example of a GUI screen displayed by the GUI unit 44 based on GUI information received by the terminal device 4. The GUI screen shown in FIG. 5 includes a map image 50, a first camera icon 51, a second camera icon 52, a first window 100, a first thumbnail image 101, and a reception component 102. Based on the GUI information, the GUI unit 44 displays the first camera icon 51 and the second camera icon 52 superimposed on the map image 50.

The first camera icon 51 and the first window 100 are associated with the GUI information received by the terminal device 4. When the terminal device 4 receives an operation by the user to select the first camera icon 51, it pops up the first window 100. The first window 100 includes a first thumbnail image 101 and a reception component 102 that displays "Watch video."

The GUI information associates the reception component 102 with the identification information of the camera 1 that captured the first thumbnail image 101. When the user performs an operation to select the reception component 102, a streaming request including the identification information is transmitted to the streaming server 32. When the streaming server 32 receives the streaming request, it transmits the corresponding streaming data to the terminal device 4.

An example of a GUI screen when the user performs an operation to select the second camera icon 52 is shown in FIG. 6. As shown in FIG. 6, a first window 100 is popped up, but does not include a thumbnail image or a reception component 102, and instead includes first hidden information 103. The first hidden information 103 displays "Live video cannot be displayed due to camera malfunction." In other words, it indicates that the latest image from the camera 1 corresponding to the second camera icon 52 cannot be displayed.

An example of a GUI screen in the conventional technology is shown in FIG. 7. In the conventional technology, the determination unit 37 does not make a determination, and the generation unit 38 does not generate GUI information. Therefore, even if a communication failure occurs between the corresponding camera 1 and the server 3, the unupdated second thumbnail image 104 and the reception component 102 are displayed as shown in FIG. 7.

When the terminal device 4 displays the GUI screen shown in FIG. 7, the user who sees this is likely to select the reception component 102. Even if the user selects the reception component 102, the video image does not start to be displayed, and therefore the user may well file a complaint with the provider of the live camera system 10.

On the other hand, in the live camera system 10 according to the present invention, if there is a possibility that images from a particular camera 1 cannot be transmitted, as shown in FIG. 6, a message is automatically displayed indicating that video images from the camera 1 will not be displayed before the user selects the camera 1, thereby reducing complaint inquiries.

Furthermore, when the malfunction of network 2 is improved and the latest thumbnail image arrives again from camera 1, the determination unit 37 determines that the difference between the update date and time and the current date and time is within a predetermined time. As a result, the generation unit 38 generates GUI information including the thumbnail image and the reception component. In other words, when the malfunction of network 2 is improved, the thumbnail image and the reception component are automatically displayed on the GUI screen, thereby reducing the burden on the provider of the live camera system 10.

Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and the components of the present invention can be modified and embodied without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above embodiment. In addition, some components may be deleted from the components of the above embodiment or the modified examples described below.

For example, the display mode of the GUI screen according to the present invention includes display modes other than the display modes described in the above embodiment. For example, FIG. 8 is a diagram showing another example of a GUI screen displayed by the GUI unit 44 based on the GUI information received by the terminal device 4 according to the present invention.

The GUI screen shown in FIG. 8 includes a map image 50, a first location icon 61, a second location icon 62, a second window 200, a third thumbnail image 201, a second non-displayed information 202, and a fourth thumbnail image 203. The GUI unit 44 displays the first location icon 61 and the second location icon 62 superimposed on the map image 50 based on the GUI information.

The second window 200 is popped up when the user selects the first location icon 61. The second window 200 includes a third thumbnail image 201, second non-display information 202, and a fourth thumbnail image 203. In other words, this indicates that three cameras 1 are installed at the location indicated by the first location icon 61, and that the images captured by each camera 1 can be selected for display on the terminal device 4.

Since the first location icon 61 and the second window 200 are associated with the GUI information, when the user performs an operation to select the first location icon 61, the second window 200 is popped up and displayed. In the second window 200, a third thumbnail image 201, second non-display information 202, and a fourth thumbnail image 203 corresponding to the three cameras 1 respectively are displayed in a list.

The third thumbnail image 201 and the fourth thumbnail image 203 are associated with the identification information of the corresponding camera 1. The second non-display information 202 is not associated with the identification information of the corresponding camera 1. The second non-display information 202 displays the message "Live video cannot be displayed due to camera malfunction."

When the user performs an operation to select the third thumbnail image 201 or the fourth thumbnail image 203, a streaming request including identification information is sent to the streaming server 32. When the streaming server 32 receives the streaming request, it transmits the corresponding streaming data to the terminal device 4. In other words, the third thumbnail image 201 and the fourth thumbnail image 203 have the same function as the reception component 102 described above.

On the other hand, since the second hidden information 202 is not associated with the identification information of the corresponding camera 1, even if the user performs an operation to select the second hidden information 202, the terminal device 4 does not accept anything and does not operate. Also, since the second hidden information 202 contains a description that a video image will not be displayed, it is expected that the user will select the third thumbnail image 201 or the fourth thumbnail image 203 without selecting the second hidden information 202.

In the above embodiment, the FTP server 31 receives thumbnail images from each camera 1, but instead of acquiring thumbnail images from each camera 1, as shown in FIG. 9, the streaming server 32 may generate thumbnail images for each camera 1 based on the video data acquired from each camera 1, and the acquired thumbnail images may be acquired by the acquisition unit 35.

In this case, the FTP server 31 is not necessary and each camera 1 only needs to generate video data, reducing the burden on each camera 1. On the other hand, the streaming server 32 has to perform more processing to generate thumbnail images from the video data of each camera 1. It is desirable to select an appropriate method depending on the operating environment of the camera 1, network 2, and server 3 that make up the system.

In the above embodiment, the determination unit 37 determines whether the acquired thumbnail image is newer than a predetermined time from the difference between the update date and time of the thumbnail image and the current date and time, but if the thumbnail image is a JPEG format image, the Exif information includes the shooting date and time, so the determination may be made using the shooting date and time instead of the update date and time. In other words, the determination unit 37 may determine whether the difference between the shooting date and time and the current date and time is within a predetermined time.

As for the judgment method, a similar judgment can be made using a method other than that described in the above embodiment. For example, since the transmission period of thumbnail images transmitted from each camera 1 to the server 3 is constant and known, the server 3 calculates a judgment date and time by adding the transmission period to the update date and time of the acquired thumbnail image, and stores the calculated judgment date and time in the storage unit 33. The server 3 may judge the malfunction of the camera 1 at the next update of the thumbnail image based on whether the absolute value of the difference between the judgment date and time and the update date and time is within a predetermined time. In this judgment, if the thumbnail image is updated at the timing of the transmission period, it is determined that the camera 1 is normal, and if not, it is determined that a malfunction has occurred in the camera 1. By making a judgment based on the absolute value, it is also possible to detect abnormalities such as transmission earlier than the default transmission period.

For example, if the transmission cycle of thumbnail images is one minute and the update date and time of the acquired thumbnail image is 15:00 on December 15, 2020, the determination date and time is 15:01 on December 15, 2020. If the next time a thumbnail image is acquired and updated, and the update date and time is 15:01:36 on December 15, 2020, the absolute value of the difference from the determination date and time will be 36 seconds. If the specified time used for determination is 30 seconds, the determination unit 37 determines that the camera 1 is malfunctioning because the absolute value of the difference is within 30 seconds, and the generation unit 38 generates GUI information including non-display information.

In addition, if the thumbnail image is a JPEG format image, the Exif information for the thumbnail image includes the shooting date and time, so the difference between the update date and time of the thumbnail image and the shooting date and time can be calculated, and if the absolute value of this difference is within a predetermined time, it can be determined that camera 1 is normal, and if it is not within the predetermined time, it can be determined that there is an abnormality in camera 1.

In the above embodiment, a GUI request is sent from the terminal device 4 to the server 3, and the server 3 sends GUI information to the terminal device 4 in a pull type manner. However, the GUI information may be sent to the terminal device 4 in a push type manner at the timing when the generation unit 38 generates the GUI information. In this case, the terminal device 4 needs to register its own address in advance. By using the push type, the terminal device 4 can display a GUI screen based on the latest GUI information.

For example, as shown in FIG. 7, the terminal device 4 displays a first window 100 including a second thumbnail image 104 and a reception component 102 as a GUI screen. A few seconds later, the communication environment between the camera 1 and the server 3 becomes unhealthy, and the server 3 generates new GUI information including hidden information. The server 3 then pushes the generated GUI information to the terminal device 4. The terminal device 4 receives the push-transmitted GUI information, and pops up the first window 100 including the first hidden information 103 as shown in FIG. 6.

By sending GUI information by push transmission from the server 3, the terminal device 4 can present the fact that the video image of the camera in question is not being displayed even if the camera 1 is unable to send images to the server 3 recently due to a malfunction of the network 2. In other words, it is possible to reduce complaints and inquiries from users.

The present invention may be a server 3 constituting a live camera system 10, or a GUI information generation method used in the server 3. The present invention may also be a program that causes the server 3 to execute a process for generating GUI information. It goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

This invention can be applied to live camera systems.

LIST OF SYMBOLS 1 Camera 1a First camera 1b Second camera 2 Network 2a First network 2b Second network 2c Third network 3 Server 4 Terminal device 4a First terminal device 4b Second terminal device 10 Live camera system 30 Web server 31 FTP server 32 Streaming server 33 Storage unit 34 Timer unit 35 Acquisition unit 36 Update unit 37 Determination unit 38 Generation unit 39 Server communication unit 41 Terminal communication unit 42 Terminal storage unit 43 Terminal control unit 44 GUI unit 50 Map image 51 First camera icon 52 Second camera icon 61 First location icon 62 Second location icon 100 First window 101 First thumbnail image 102 Reception component 103 First non-displayed information 104 Second thumbnail image 200 Second window 201 Third thumbnail image 202 Second non-display information 203 Fourth thumbnail image

Claims (5)

A live camera system including a plurality of cameras which sequentially capture images, a server which receives moving image data generated based on the images from each of the cameras via a network, and a terminal device which sequentially obtains streaming data converted from the moving image data from the server and is capable of displaying the images,
The server,
an acquisition unit that sequentially acquires thumbnail images obtained by reducing the images captured by each of the cameras;
The thumbnail image includes identification information for identifying the camera that captured the image,
an update unit that updates the thumbnail image having the same identification information when the acquisition unit acquires the thumbnail image, and stores the updated image in association with an update date and time indicating a date and time of the update;
a determination unit that determines, at a predetermined timing, whether a difference between the update date and time associated with the updated thumbnail image and a current date and time is within a predetermined time;
a generation unit that generates GUI information including the thumbnail image and an acceptance component that accepts a transmission request of the streaming data of the camera identified by the identification information associated with the thumbnail image when the determination unit determines that the difference is within the predetermined time, and generates the GUI information including non-display information indicating that the latest image of the camera identified by the identification information associated with the thumbnail image cannot be displayed when the determination unit determines that the difference is not within the predetermined time, instead of the thumbnail image;
a server communication unit that transmits the GUI information to the terminal device when a GUI request for the GUI information is received from the terminal device via the network;
The terminal device
a terminal communication unit that transmits the GUI request to the server and receives the GUI information from the server;
a GUI unit that displays a GUI screen including the thumbnail image and the reception component or the non-display information based on the GUI information and receives a selection for the reception component.
Live camera system. the GUI information includes a map image and a camera icon indicating a position where each of the cameras is installed;
the GUI unit displays a plurality of the camera icons superimposed on the map image based on the GUI information, and when a selection of any one of the plurality of the camera icons is accepted, displays the thumbnail image of the camera installed at the position indicated by the camera icon and the acceptance part, or the non-display information.
The live camera system according to claim 1. the GUI information includes a map image and a location icon indicating a location where each of the cameras is installed;
the GUI unit displays the plurality of point icons superimposed on the map image based on the GUI information, and when a selection of any one of the plurality of point icons is accepted, displays a list including the thumbnail images of the cameras installed at the points indicated by the point icons and the reception parts, or the non-display information, based on the GUI information.
The live camera system according to claim 1. The thumbnail image further includes a shooting date and time indicating the date and time when the image was shot.
The determination unit uses the shooting date and time instead of the update date and time.
The live camera system according to any one of claims 1 to 3. A server receives, via a network, moving image data generated based on images from a plurality of cameras that sequentially capture images, and when a transmission request for streaming data converted from the moving image data is received from a terminal device, the server sequentially transmits the streaming data to the terminal device via the network,
an acquisition unit that sequentially acquires thumbnail images obtained by reducing the images captured by each of the cameras;
The thumbnail image includes identification information for identifying the camera that captured the image,
an update unit that updates the thumbnail image having the same identification information when the acquisition unit acquires the thumbnail image, and stores the updated image in association with an update date and time indicating a date and time of the update;
a determination unit that determines, at a predetermined timing, whether a difference between the update date and time associated with the updated thumbnail image and a current date and time is within a predetermined time;
a generation unit that generates GUI information including the thumbnail image and an acceptance component that accepts a transmission request for the streaming data of the camera identified by the identification information associated with the thumbnail image when the determination unit determines that the difference is within the predetermined time, and generates the GUI information including non-display information indicating that the latest image of the camera identified by the identification information associated with the thumbnail image cannot be displayed when the determination unit determines that the difference is not within the predetermined time, instead of the thumbnail image;
a server communication unit that transmits the GUI information to the terminal device when a GUI request for the GUI information is received from the terminal device via the network.

Images