JP7742382B2 - Relay device, relay method, relay program, and relay system - Google Patents

Description

The present invention relates to a relay device, a relay method, a relay program, and a relay system.

In recent years, there has been an increase in the use of AI (Artificial Intelligence) cameras, which combine imaging devices such as cameras for capturing video with computer systems for analyzing the captured video data. AI cameras are equipped with a GPU that performs high-speed computational processing such as deep learning-based AI, and a wireless or wired communication module. By using such AI cameras, processing such as feature detection and annotation from video data can be performed on the AI camera itself, which is expected to improve work efficiency.

One configuration of AI cameras is one in which video analysis processing is performed by a computer at the edge of the network. AI cameras that adopt this configuration include those equipped with an EAB (Edge Artificial-Intelligence Box), which processes video data using AI and then transmits it via the network to a server or other device running an application.

In such AI cameras, the EAB acquires video data from a camera connected to the user's network environment, and the results of AI processing within the EAB are sent to a server, etc., via the Internet. However, since each user's camera device and network environment are different, it is desirable to carry out communication between the EAB and the camera in the optimal network environment based on information about the camera device and the video being generated.

Incidentally, a video data distribution technology has been proposed in which a camera generates low-quality data from the same video signal and records it on a management server, while recording high-quality data on the camera's own recording device, and then switches between the low-quality data and high-quality data and distributes them in response to user operation (see, for example, Patent Document 1).

JP 2016-58994 A

However, image quality (such as resolution) and communication environments differ for each imaging device, and selecting an appropriate network environment for each imaging device by switching communication paths in response to user instructions is a heavy workload and difficult to achieve. Furthermore, if a communication path is selected that can handle high-quality data, network resources will be wasted for imaging devices that transmit low-quality data. Conversely, if a communication path that supports low-quality data is selected in order to maintain the network environment, the transmission load for imaging devices that transmit high-quality data may become too high, making communication itself difficult. As such, conventional relay technologies risk wasting network resources and increasing network load, making it difficult to improve communication efficiency.

The disclosed technology has been developed in light of the above, and aims to provide a relay device, relay method, relay program, and relay system that improve communication efficiency.

In order to solve the above-mentioned problems and achieve the object, the relay device of the present invention has the following units: A first route selection unit selects a first communication route from among multiple communication routes between the relay device and the image capture device based on information about the image capture device. A first communication unit acquires video data from the image capture device using the first communication route selected by the first route selection unit. A processing unit performs first processing using the video data acquired by the first communication unit. A second communication unit transmits the video data that has been subjected to the first processing by the processing unit to a data processing device using the second communication route.

This invention makes communication more efficient.

FIG. 1 is a system configuration diagram of a relay system according to an embodiment. FIG. 2 is a diagram showing an example of a connection between an EAB and an imaging device via a customer LAN. FIG. 3 is a diagram showing an example of a direct connection of the EAB with an imaging device. FIG. 4 is a block diagram of an EAB according to an embodiment. FIG. 5 is a diagram illustrating an example of communication path selection. FIG. 6 is a diagram showing an example of communication path selection accompanied by a change in resolution setting. FIG. 7 is a diagram illustrating EAB communication settings in response to a connection with a video analysis server. FIG. 8 is a flowchart of a process for determining a communication path by EAB according to the embodiment. FIG. 9 is a diagram showing the hardware configuration of a computer.

Embodiments of the relay device, relay method, relay program, and relay system disclosed in the present application are described in detail below with reference to the accompanying drawings. Note that the relay device, relay method, relay program, and relay system disclosed in the present application are not limited to the following embodiments.

(Embodiment)
(Introduction)
The system acquires video from cameras connected to the client's network environment, analyzes the data within the device, and transmits the results via the Internet. The client's network environment is already established and operational, and there were concerns about the impact of adding new video analysis data. Therefore, a relay device was used to simultaneously transmit multiple data using different communication types. However, because the resolution of each camera varies, it is difficult to automatically optimize the communication between the relay device and the camera in the optimal network environment.

The relay device according to the embodiment prioritizes and categorizes communication routes based on the amount of data being communicated and information obtainable from the destination camera device, and determines the communication route to use without requiring user operation. The relay device according to the embodiment is described in detail below.

(System configuration)
1 is a system configuration diagram of a relay system according to an embodiment. The relay system 100 includes an EAB 1, which is a relay device, a customer LAN (Local Area Network) 2, a video analysis server 3 on the cloud, and a terminal device 4.

Customer LAN 2 is equipped with IP (Internet Protocol) cameras 21-23, which are image capture devices, a customer terminal device 201, a hub 202, and other components. EAB 1 is also connected to customer LAN 2.

IP cameras 21-23 are connected to EAB1 via customer LAN 2. IP cameras 21-23 are connected to EAB1 via multiple communication paths. IP cameras 21-23 may connect to EAB1 via hub 202. For example, IP cameras 21-23 can each communicate with EAB1 using a wired connection such as Ethernet (registered trademark), a wireless connection such as Wi-Fi, or a wireless connection such as private 5G. However, these communication paths are merely examples, and other communication paths can also be used as long as they comply with standards that allow data transfer.

The specifications for capturing and transferring data are predetermined for each of the IP cameras 21-23. For example, the resolution, image size, frame rate (fps: frames per second), and video data codec that can be used are predetermined for each of the IP cameras 21-23. Here, each of the IP cameras 21-23 may be able to use multiple types of settings. For example, the IP camera 21 may be able to use multiple resolutions.

The settings used for image capture and data transfer for IP cameras 21-23 are specified in advance by the administrator. IP cameras 21-23 store image capture device information that registers the manufacturer name, model name, and other information that can be used by each image capture device 20, such as resolution, image size, frame rate, and codec.

Each of the IP cameras 21-23 captures video according to the specified settings. Furthermore, when the IP cameras 21-23 receive an instruction from the EAB 1 to change the resolution settings, they change the settings to the specified resolution and continue capturing video. The IP cameras 21-23 then transmit the video data of the captured video to the EAB 1 using the communication path specified by the EAB 1.

Note that while IP cameras 21-23 have been used as an example of the imaging devices in this description, the imaging devices may be other devices that can capture video.

The customer terminal device 201 can be connected to the IP cameras 21-23 via a hub 202 or the like. The administrator of the IP cameras 21-23 can use the customer terminal device 201 to configure the IP cameras 21-23.

The EAB 1 is connected to IP cameras 21-23 via the customer LAN 2. Figure 2 shows an example of the connection of the EAB to a camera via the customer LAN 2. For example, as shown in Figure 2, the EAB 1 has a mobile data communication interface 101, a wired communication interface 102, and a Wi-Fi communication interface 103.

The EAB 1 can communicate with the IP cameras 21-23 via the customer LAN 2 using one or more of the mobile data communication interface 101, the wired communication interface 102, and the Wi-Fi communication interface 103. For example, in FIG. 2, the EAB 1 is connected to the customer LAN 2 using the Wi-Fi communication interface 103. In this case, the EAB 1 can communicate with the IP cameras 21-23 using a communication path via Wi-Fi communication.

Although Figure 2 shows a state in which the EAB 1 is connected via one route, the EAB 1 may be connected to the customer LAN 2 via multiple interfaces, including the mobile data communication interface 101, the wired communication interface 102, and the Wi-Fi communication interface 103. In this case, the EAB 1 can select a communication route from the connected routes to communicate with the IP cameras 21-23.

Figure 3 shows an example of a direct connection between an EAB and a camera. For example, as shown in Figure 3, the EAB 1 may be directly connected to the IP camera 21 via a network. In Figure 3, the EAB 1 is connected to the IP camera 21 using a wired communication interface 102. However, the EAB 1 may be connected to the IP camera 21 via a communication path using another interface depending on the network capabilities of the IP camera 21, or may be connected via multiple communication paths.

Returning to Figure 1, the explanation continues. From among the multiple communication paths connecting each of the IP cameras 21-23, the EAB1 identifies the optimal type of communication path to use for sending and receiving video data for each of the IP cameras 21-23, based on the resolution of the video generated by each of the IP cameras 21-23. The EAB1 then notifies each of the IP cameras 21-23 of the identified type of communication path as the communication path to use for transmitting the video data. If the identified type of communication path is unavailable, the EAB1 determines an available communication path corresponding to a lower resolution as the communication path to use. The EAB1 then instructs the IP cameras 21-23 to change their settings to transmit video data via the communication path determined to be used, and notifies them of the communication path to use for transmitting the video data.

The EAB1 also performs AI processing using a specified AI application on each piece of video data transmitted from the IP cameras 21 to 23. Here, the EAB1 can use the same AI application to process each piece of data, or different applications.

Furthermore, EAB1 determines, according to the specified priority, the communication path to be used for transmitting the video data and the results of the AI processing from among multiple communication paths connecting to the video analysis server 3 on the cloud. EAB1 then transmits the results of the AI processing and the video data to the video analysis server 3 using the determined communication path. EAB1 will be described in detail later.

The video analysis server 3 is a device deployed on the cloud. The video analysis server 3 is connected to the EAB1 via the internet. The video analysis server 3 receives the results of the AI processing and the video data from the EAB1. The video analysis server 3 then performs processing to add value to the video data, for example by visualizing the results of the AI processing. The video analysis server 3 then transmits the processed video data to the terminal device 4. This video analysis server 3 is an example of a "data processing device," and the processing by the video analysis server 3 using the video data and the results of the AI processing is an example of a "second process."

Terminal device 4 is a device that performs various controls using images captured by IP cameras 21-23. For example, terminal device 4 uses images captured by IP cameras 21-23 to perform control processes to realize smart retail, smart buildings, smart cities, smart factories, etc.

(Composition of EAB)
FIG. 4 is a block diagram of an EAB according to an embodiment. Next, details of the EAB 1 according to this embodiment will be described with reference to FIG. 4. Here, devices for generating images, such as IP cameras 21 to 23, will be described as an image capturing device 20 without distinguishing between them. As shown in FIG. 2, the EAB 1 has a first communication unit 11, an AI processing unit 12, a second communication unit 13, a first route selection unit 14, and a second route selection unit 15.

The first communication unit 11 is connected to one or more image capture devices 20 via multiple communication paths. The multiple communication paths between the first communication unit 11 and each image capture device 20 are examples of "first multiple communication paths." For example, the first communication unit 11 has a communication path for a wired connection via Ethernet communication, a communication path for a wireless connection via Wi-Fi communication, and a communication path for a wireless connection via private 5G communication. However, the communication paths available between the first communication unit 11 and each image capture device 20 do not have to be the same.

The first communication unit 11 receives camera device information including information on the image quality of the video data generated by the camera device 20. For example, when the EAB 1 connects to the customer LAN 2, the first communication unit 11 obtains camera device information, which includes information on settings such as resolution, from each camera device 20 connected to the customer LAN 2 using one of the available communication paths.

Then, the first communication unit 11 outputs the camera device information to the first route selection unit 14. After that, the first communication unit 11 receives information about the communication route to be used by the camera device 20 when transmitting video data from the first route selection unit 14 and transmits this information to the camera device 20.

Furthermore, when the first communication unit 11 receives an instruction to change the resolution setting from the first path selection unit 14, it transmits the instruction to change the resolution setting to the targeted image capture device 20, causing the image capture device 20 to change its resolution setting. Then, the first communication unit 11 receives information on the communication path corresponding to the changed resolution from the first path selection unit 14 as information on the communication path to be used by the image capture device 20 when transmitting video data, and transmits this information to the image capture device 20.

Then, the first communication unit 11 receives the video data from the imaging device 20 via the communication path specified by the first path selection unit 14. The first communication unit 11 then outputs the received video data to the AI processing unit 12.

The first route selection unit 14 receives input of camera information for each camera device 20 from the first communication unit 11. In addition to the manufacturer name and model name, the camera information registers information indicating the image quality that each camera device 20 can use, such as resolution, image size, frame rate, and codec. The first route selection unit 14 performs the following route selection process for each camera device 20. Because the route selection process is performed in the same way for all camera devices 20, the route selection process will be explained below using one camera device 20 as an example.

The first path selection unit 14 acquires the resolution of the image capture device 20 from the image capture device information. Here, the first path selection unit 14 holds information on the type of communication path to be used for each resolution. For example, the first path selection unit 14 divides the resolution into three categories (high, medium, and low) according to height, and holds information on the type of communication path to be used for each category. Specifically, the first path selection unit 14 holds information such as using a wired communication path for high resolution, a private 5G communication path for medium resolution, and a Wi-Fi communication path for low resolution. However, this division of resolution is not limited to three levels, and may be two levels or four or more levels, and the type of communication path corresponding to each resolution may be determined.

The first path selection unit 14 identifies the type of communication path according to the resolution of the camera device 20 obtained from the camera device information. Next, the first path selection unit 14 determines whether communication with the camera device 20 is possible via the identified type of communication path. For example, if a communication path of the identified type does not exist between the camera device 20 and the camera device 20, or if a communication path of the identified type exists but is difficult to use due to a failure, the first path selection unit 14 determines that communication with the camera device 20 is not possible via the identified type of communication path.

If communication is possible, the identified type of communication path is determined as the communication path to be used for sending and receiving video data from the imaging device 20. The first path selection unit 14 then notifies the imaging device 20 via the first communication unit 11 of information about the communication path to be used by the imaging device 20 for sending the video data.

Figure 5 is a diagram showing an example of communication path selection. For example, the following describes a case in which the EAB1 is connected to each of the IP cameras 21-23 via a wired communication path 211, a Wi-Fi communication path 212, and a private 5G communication path 213. The following also describes a case in which the first path selection unit 14 divides resolution into three categories: high, medium, and low, and uses communication paths 211, 213, and 212 in descending order of resolution.

The first path selection unit 14 confirms from the imaging device information of the IP camera 21 that the IP camera 21 uses a resolution included in the high category. The first path selection unit 14 then identifies the wired communication path 211, which corresponds to the high category, as an appropriate communication path for transmitting and receiving video data to and from the IP camera 21. The first path selection unit 14 also confirms from the imaging device information of the IP camera 22 that the IP camera 22 uses a resolution included in the medium category. The first path selection unit 14 then identifies the private 5G communication path 213, which corresponds to the medium category, as an appropriate communication path for transmitting and receiving video data to and from the IP camera 22. The first path selection unit 14 also confirms from the imaging device information of the IP camera 23 that the IP camera 23 uses a resolution included in the low category. The first path selection unit 14 then identifies the Wi-Fi communication path 212, which corresponds to the low category, as an appropriate communication path for transmitting and receiving video data to and from the IP camera 23. Since communication with IP cameras 21-23 is possible using any of these communication paths, the first path selection unit 14 determines each of these types of communication paths as the communication path to use. As a result, the first communication unit 11 communicates with IP cameras 21-23 using a communication path that corresponds to each of their resolutions, as shown in FIG. 5.

Returning to Figure 4, the explanation continues. On the other hand, if communication is not possible via the specified type of communication path, for example, because the specified type of communication path is not connected to the image capture device 20 or is unavailable due to a failure, the first path selection unit 14 determines whether communication with the image capture device 20 is possible via a communication path that supports a lower resolution. Specifically, the first path selection unit 14 determines whether communication with the image capture device 20 is possible via a communication path that supports a lower resolution, and whether the image capture device 20 supports that resolution.

If a communication path corresponding to a lower resolution is available, the first path selection unit 14 determines the communication path to use as the communication path with the highest possible resolution that can be used by the camera device 20 among the resolutions corresponding to the available communication paths. This communication path is the communication path corresponding to the highest possible resolution that can be used by the camera device 20 among the resolutions corresponding to the communication paths that the EAB1 can use between the camera device 20 and the EAB1. Next, the first path selection unit 14 sends an instruction to the camera device 20 via the first communication unit 11 to change the setting to the highest possible resolution that can be used by the camera device 20 among the resolutions corresponding to the determined communication path. Thereafter, the first path selection unit 14 notifies the camera device 20 via the first communication unit 11 of information about the communication path to be used by the camera device 20 to transmit video data.

Figure 6 shows an example of communication path selection involving a change in resolution settings. For example, let us consider a case where EAB1 is connected to each of IP cameras 21-23 via Wi-Fi communication path 212, and wired communication path 211 and private 5G communication path 213 are unavailable due to a failure or other reason. Again, the first path selection unit 14 divides the resolution into three categories: high, medium, and low, and uses communication paths 211, 213, and 212 in descending order of resolution.

In this case, too, the first path selection unit 14 confirms from the imaging device information of the IP camera 21 that the IP camera 21 uses a resolution included in the high category. The first path selection unit 14 then determines whether the communication path 211 corresponding to the high category is available. In this case, the first path selection unit 14 determines that the communication path 211 is unavailable and that the communication path 212 using Wi-Fi communication is available. The first path selection unit 14 then decides to use the communication path 212 using Wi-Fi, which corresponds to the low category, for sending and receiving video data to and from the IP camera 21. The first path selection unit 14 then instructs the IP camera 21 to change the setting to a resolution that is available for the IP camera 21 and is as high as possible, from among the resolutions available on the communication path 212. The first path selection unit 14 also confirms from the imaging device information of the IP camera 22 that the IP camera 22 uses a resolution included in the medium category. The first path selection unit 14 then determines whether the communication path 213 corresponding to the medium category is available. In this case, the first path selection unit 14 determines that communication path 213 is unavailable and that communication path 212 using Wi-Fi communication is available. The first path selection unit 14 then decides to use communication path 212 using Wi-Fi communication, which corresponds to the low category, for sending and receiving video data to and from IP camera 22. The first path selection unit 14 then instructs IP camera 22 to change the resolution setting to a resolution that is available for IP camera 22 and is as high as possible, among the resolutions available on communication path 212. The first path selection unit 14 also confirms from the imaging device information of IP camera 23 that IP camera 23 uses a resolution included in the low category. In this case, because communication path 211 using Wi-Fi, which corresponds to the low category, is available, the first path selection unit 14 decides to use communication path 211 for sending and receiving video data to and from IP camera 23. As shown in FIG. 5, the first path selection unit 14 then communicates with all of IP cameras 21 to 23 using communication path 212.

For example, the communication path 211 corresponding to the resolution of the IP camera 21 is an example of a "first communication path," and the inability to communicate between the IP camera 21 and the EAB1 via the first communication path is an example of a "case where it is difficult to select the first communication path." Furthermore, the communication path 212 selected by the first path selection unit 14, other than the communication path 211 suitable for the resolution of the IP camera 21, is an example of a "third communication path." In other words, when it is difficult to select the communication path 211, which is the first communication path based on the information from the IP camera 21, the first path selection unit 14 selects the communication path 212, which is the third communication path other than the communication path 211 suitable for the resolution of the IP camera 21, from the communication paths with the imaging device 20, and changes the settings of the IP camera 21 so that video data is transmitted via the third communication path.

In this embodiment, the resolution categories are associated one-to-one with different types of communication paths 211-213, and the first path selection unit 14 selects a communication path corresponding to the resolution. Furthermore, in order to improve the image quality of the video data, the first path selection unit 14 changes the settings of the image capture device 20 so that the highest possible resolution can be used among the available communication paths. However, the methods for selecting a communication path for a resolution and changing the resolution settings of the image capture device 20 are not limited to this.

For example, the resolution categories can be associated with the communication paths 211-213 as follows: The communication path 211 using wired communication can transmit video data in all resolution categories, high, medium, and low. The communication path 212 using private 5G communication can transmit video data in two resolution categories, medium and low. In contrast, the communication path 213 using Wi-Fi communication can only transmit video data in the low resolution category. In this case, the first path selection unit 14 may determine the communication path that the image capture device 20 will use to transmit video data as follows:

For example, if the resolution of the image capture device 20 is in the low category, all of the communication paths 211 to 213 are available, and the network resources of the wired communication path 211 are greater than those of the communication paths 212 and 213, the first path selection unit 14 may use the wired communication path 211 to transmit video data from the image capture device 20 in order to make effective use of network resources. In this case, selecting the communication path 211 based on the resolution and network resources of the image capture device 20 is an example of "selecting a first communication path based on information about the image capture device."

In this case, if multiple image capture devices 20 with low resolution are connected to EAB1 and all communication paths 211-213 are available, connecting all image capture devices 20 to wired communication path 211 may place a load on the network. Therefore, if the communication speed of communication path 211 falls below a threshold, the first path selection unit 14 may change to using communication path 212 with private 5G communication for transmitting video data from some image capture devices 20 so that the communication speed falls within the threshold. Similarly, if the communication speed of communication path 212 falls below a threshold, the first path selection unit 14 may change to using communication path 213 with Wi-Fi communication for transmitting video data from some image capture devices 20 so that the communication speed falls within the threshold.

The same applies when the resolution of the image capture device 20 is in the high category. If the communication speed of the communication path 211 falls below the threshold, the first path selection unit 14 may change the communication path 212 using private 5G communication to transmit video data from several image capture devices 20 so that the communication speed falls within the threshold. In this case, the first path selection unit 14 instructs the target image capture device 20 to change its resolution setting to the medium category in order to use the communication path 212 using private 5G communication.

When the communication speed falls below the threshold, this is an example of "when it is difficult to select the first communication path." Communication path 212, selected instead of communication path 211, is an example of a "third communication path."

Furthermore, in this embodiment, the first path selection unit 14 determines the communication path according to the resolution, which determines the data volume of the video data generated by the image capture device 20. However, the criteria for determining the data volume to determine the communication path are not limited to this, and other information can be used as long as it allows for understanding the amount of data to be transmitted. For example, the data volume can also be determined by other information that indicates the image quality of the video, such as image size, frame rate (FPS), bit rate, or codec. Therefore, the first path selection unit 14 may obtain such information that indicates the image quality of the video from the image capture device information and determine the communication path to use.

Here, the first route selection unit 14 may identify this information from the manufacturer name and model name of the imaging device 20 registered in the imaging device information. The first route selection unit 14 may also combine some of this information and use it to determine the communication route.

Furthermore, the first path selection unit 14 may acquire the radio wave strength of wireless communication with the image capture device 20 and use it to select a communication path, for example, by not using wireless communication with radio wave strength below a reference value. Also, the first path selection unit 14 may use the type of AI application corresponding to the image capture device 20, which is used for AI processing in the AI processing unit 12 for each piece of video data, to select a communication path.

For example, when receiving video data to be processed by an AI application that can tolerate a slight drop in data, the first path selection unit 14 can select a communication path that allows a small amount of data to be transferred. In other words, the first path selection unit 14 can select a communication path based on the video quality required for the AI application to maintain a minimum level of performance.

In this way, the first path selection unit 14 can determine the communication path to use for transmitting video data to and from the camera device 20 based on various pieces of information about the camera device 20, such as information indicating the image quality of the video, radio wave strength, and the type of AI application. Here, the first path selection unit 14 selected a communication path corresponding to the information acquired based on a predetermined correspondence, but this is not limiting and the communication path may also be determined using AI, for example. The communication path selected by the first path selection unit 14 for transmitting video data between the first communication unit 11 and the camera device 20 is an example of a "first communication path."

Returning to Figure 4, we will continue the explanation. The AI processing unit 12 receives input of video data transmitted from each of the multiple image capture devices 20 from the first communication unit 11. The AI processing unit 12 has multiple AI applications. For example, one AI application may be an application for detecting human faces, and another AI application may be an application for detecting automobiles, and each AI application may perform different AI processing.

The AI processing unit 12 performs AI processing on each piece of acquired video data using a specified AI application. The AI processing unit 12 may retain predetermined information for each image capture device 20 as the designation of the AI application to be used, or may acquire the designation of the application to be used from information added to the video data. The AI processing unit 12 then outputs the results of the AI processing together with the video data to the second communication unit 13. This AI processing unit 12 is an example of a "processing unit," and the AI processing of the video data by the AI processing unit 12 is an example of a "first process."

The second path selection unit 15 receives input specifying the priority of each of the multiple communication paths connecting the second communication unit 13 and the video analysis server 3 from an input unit (not shown). These multiple communication paths connecting the second communication unit 13 and the video analysis server 3 are an example of "second multiple communication paths." The second path selection unit 15 then determines the communication path with the highest priority among the available communication paths as the communication path to be used for communication with the video analysis server 3. The communication path selected by the second path selection unit 15 to be used for transmitting video data and AI processing results between the second communication unit 13 and the video analysis server 3 is an example of a "second communication path."

For example, as shown in FIG. 5, the EAB 1 will be described as having a communication path 301 using a wired connection using Ethernet, a communication path 302 using 5G communication, a communication path 303 using LTE (Long Term Evolution) communication, and a communication path 304 using Wi-Fi. The second path selection unit 15 receives designation of the communication paths in order of decreasing priority, for example, communication path 301, communication path 302, communication path 303, and communication path 304. In this case, the second path selection unit 15 determines the wired communication path 301, which has the highest priority among the available communication paths, as the communication path to be used for communication with the video analysis server 3. As a result, the EAB 1 communicates with the video analysis server 3 using the wired communication path 301.

Returning to Figure 4, the explanation continues. The second path selection unit 15 instructs the second communication unit 13 to communicate with the video analysis server 3 using the communication path that has been decided to be used for communication. The second path selection unit 15 then sets the communication path that has been decided to be used for communication with the video analysis server 3 to on in the communication settings of the second communication unit 13, and sets other communication paths to off. However, when the first communication unit 11 and the second communication unit 13 communicate using the same communication interface, the second path selection unit 15 maintains the on setting for the communication path used by the first communication unit 11.

Figure 7 shows the communication settings of the EAB in response to connection with the video analysis server. Here, we will explain the case where the EAB 1 has a mobile data communication interface 101, a wired communication interface 102, and a Wi-Fi communication interface 103 as communication interfaces. We will also assume that the wired communication interface 102 is used for communication between the EAB 1 and the customer LAN 2.

Connection status 111 indicates a state in which the EAB 1 and the video analysis server 3 communicate via mobile data communication. In this case, the second route selection unit 15 turns on the setting for communication via the mobile data communication interface 101, leaves the setting for communication via the wired communication interface 102 on, and turns off the setting for communication via the Wi-Fi communication interface 103.

Connection status 112 indicates a state in which the EAB 1 and the video analysis server 3 communicate via Wi-Fi. In this case, the second route selection unit 15 turns off communication via the mobile data communication interface 101, leaves communication via the wired communication interface 102 on, and turns on communication via the Wi-Fi communication interface 103.

Connection status 112 indicates a state in which EAB1 and the video analysis server 3 communicate via wired communication. In this case, the second path selection unit 15 leaves the setting for communication via the wired communication interface 102 on, and turns off the setting for communication via the wired communication interface 102 and the Wi-Fi communication interface 103.

Returning to Figure 4, the explanation continues. The second communication unit 13 is connected to the video analysis server 3 via one or more communication paths. For example, the second communication unit 13 may have a communication path for a wired connection using Ethernet communication, a communication path for a wireless connection using 5G communication, a communication path for wireless communication using LTE communication, and a communication path for a wireless connection using Wi-Fi.

The second communication unit 13 receives from the second route selection unit 15 a specification of the communication route to be used for communication with the video analysis server 3. The second communication unit 13 then transmits the video data and the results of the AI processing to the video analysis server 3 using the communication route specified by the second route selection unit 15. At this time, the second communication unit 13 can transmit each piece of video data and the results of the AI processing to the video analysis server 3 together using the same communication route.

(Relay processing)
8 is a flowchart of a process of determining a communication path by EAB according to the embodiment. Next, a flow of a process of determining a communication path by EAB 1 according to the embodiment will be described with reference to FIG.

The first route selection unit 14 acquires the camera information for each camera device 20 connected to the customer LAN 2 (step S1).

Next, the first route selection unit 14 selects one imaging device 20 (step S2).

Next, the first route selection unit 14 identifies a communication route suitable for sending and receiving video data to and from the selected camera device 20 based on the acquired camera device information (step S3).

Next, the first route selection unit 14 determines whether the identified communication route exists as a communication route connecting with the image capture device 20 and is usable as a communication route to be used for sending and receiving video data (step S4).

If the identified communication route is available for use (step S4: Yes), the first route selection unit 14 determines the identified communication route as the communication route to be used (step S5). The communication route determination process then proceeds to step S8.

On the other hand, if it is difficult to use as a communication path to be used (step S4: No), the first path selection unit 14 extracts the highest possible resolution that can be used by the selected image capture device 20 from among the resolutions corresponding to the available communication paths between the image capture device 20 and the selected image capture device 20. Then, the first path selection unit 14 determines the communication path corresponding to the extracted resolution from the available communication paths as the communication path to be used (step S6).

Then, the first route selection unit 14 notifies the image capture device 20 via the first communication unit 11 of an instruction to change the settings to the resolution corresponding to the determined communication route, thereby changing the settings of the image capture device 20 (step S7). After that, the communication route determination process proceeds to step S8.

Then, the first route selection unit 14 notifies the imaging device 20 of the communication route that has been determined to be used (step S8).

Next, the first route selection unit 14 determines whether communication routes have been determined for all camera devices 20 connected to the customer LAN 2 (step S9). If there are any camera devices 20 for which communication routes have not been determined (step S9: No), the communication route determination process returns to step S2.

On the other hand, if communication paths have been determined for all of the image capture devices 20 (step S9: Yes), the first path selection unit 14 completes setting up communication paths with the image capture devices 20. Next, the second path selection unit 15 acquires from the input unit the priority of the communication paths to be used for sending and receiving video data and AI processing results with the video analysis server 3 (step S10).

Then, the second route selection unit 15 determines the communication route to be used with the video analysis server 3 according to the priority. The second route selection unit 15 then configures the communication settings of the second communication unit 13 so that video data is sent and received with the video analysis server 3 using the determined communication route (step S11).

The first communication unit 11 receives video data from each image capture device 20 via the determined communication path between the image capture device 20 and each image capture device 20 (step S12).

The AI processing unit 12 performs AI processing on the video data received by the first communication unit 11 (step S13).

The second communication unit 13 transmits the video data received from each image capture device 20 and the results of each AI processing to the video analysis server 3 via the determined communication path between the image capture device 20 and the video analysis server 3 (step S13).

(effect)
As described above, the EAB 1 according to this embodiment determines the type of communication path to use based on the amount of data determined by the image quality, such as the resolution, of the video data generated by the image capture device 20. This makes it possible to select the optimal communication path for each image capture device 20, automatically suppressing the waste of network resources and the increase in network load, and improving communication efficiency.

Furthermore, when the optimal communication route determined based on the amount of video data being sent cannot be used, the EAB1 according to this embodiment changes the settings of the imaging device 20 to reduce the amount of video data, and selects a suitable communication route from among the communication routes with reduced data volume as the next best communication route. This makes it possible to minimize waste of network resources and increases in network load, and improve communication efficiency, even when the communication route considered optimal cannot be used.

That is, EAB1, which is a relay device according to the embodiment, has a first route selection unit 14 that selects a first communication route from among a plurality of first communication routes between the image capture device 20 and the image capture device 20 based on information about the image capture device 20, a first communication unit 11 that acquires video data from the image capture device 20 using the first communication route selected by the first route selection unit 14, an AI processing unit 12 that performs first processing using the video data acquired by the first communication unit 11, and a second communication unit 13 that transmits the video data that has undergone the first processing by the AI processing unit 12 to the data processing device using the second communication route.

This allows the EAB1 to automatically select the optimal communication route to receive video data from the imaging device 20, automatically reducing the waste of network resources and the increase in network load, thereby improving communication efficiency.

In EAB1, which is a relay device according to this embodiment, the first route selection unit 14 selects the first communication route based on information specifying the amount of video data included in the information from the imaging device 20.

This allows EAB1 to select an appropriate communication path depending on the amount of data, ensuring efficient communication.

In EAB1, which is a relay device according to this embodiment, the first path selection unit 14 selects the first communication path based on information representing image quality, which is information that defines the data volume of video data.

This allows EAB1 to select an appropriate communication path based on the amount of data and image quality, ensuring more efficient communication.

The EAB1, which is a relay device according to this embodiment, further includes a second path selection unit 15 that selects a second communication path based on the priority of each of multiple second communication paths between the EAB1 and the video analysis server 3, which is a data processing device, and the second communication unit 13 transmits the video data using the second communication path selected by the second path selection unit 15.

This allows the EAB1 to automatically select the optimal communication route and send video data and AI processing results to the video analysis server 3, automatically reducing the waste of network resources and increasing network load, and improving communication efficiency.

(program)
It is also possible to create a program written in a computer-executable language that describes the processes performed by the EAB 1 described in the above embodiment. In this case, the same effects as those of the above embodiment can be achieved by having a computer execute the program. Furthermore, such a program may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read and executed by a computer to achieve the same processes as those of the above embodiment.

Figure 9 is a diagram showing the hardware configuration of a computer. EAB1 is realized, for example, by a computer 90 shown in Figure 9. As shown in Figure 9, the computer 90 has, for example, a processor 91, memory 92, a hard disk 93, and a communication interface 94.

The processor 91 operates based on programs stored on the hard disk 93 and controls each component. The memory 92 stores, for example, a boot program executed by the processor 91 when the computer 90 starts up. The hard disk 93 stores the programs executed by the processor 91 and data used by those programs.

The communication interface 94 receives data from other devices via the communication network NW (Network) and sends it to the processor 91, and transmits data generated by the processor 91 to other devices via the communication network NW. The communication interface 94 realizes the functions of the first communication unit 11 and the second communication unit 13 illustrated in Figure 4.

The processor 91 reads various programs stored on the hard disk 93, expands them into the memory 92, and executes them to realize the functions of the AI processing unit 12, first route selection unit 14, and second route selection unit 15 illustrated in FIG. 4. The processor 91 reads and executes programs from the hard disk 93, but as another example, it may also obtain these programs from other devices via the communication network NW.

(others)
Although various embodiments have been described in detail herein with reference to the drawings, these embodiments are merely examples and are not intended to limit the present invention. The features described herein can be realized in various ways, including various modifications and improvements based on the knowledge of those skilled in the art.

Furthermore, the above-mentioned "module (-er suffix, -or suffix)" can be interpreted as a unit, means, circuit, step, etc. For example, a communication module, a processing module, and a selecting module can be interpreted as a communication unit, a control unit, and a memory unit, respectively.

1 EAB
2 Customer LAN
3 Video analysis server 4 Terminal device 11 First communication unit 12 AI processing unit 13 Second communication unit 14 First route selection unit 15 Second route selection unit 20 Imaging device 21 to 23 IP camera 100 Relay system 201 Customer terminal device 202 Hub

Claims (9)

a first path selection unit that selects, for each of a plurality of image capture devices, a first communication path from among a plurality of communication paths between the image capture device and the respective image capture devices based on information about the image capture device, and when a plurality of image capture devices have selected the same specific communication path as the first communication path and the communication speed of the specific communication path falls below a threshold, reselects, for at least one of the image capture devices for which the specific communication path has been selected, a communication path having a lower transfer rate than the specific communication path as the first communication path;
a first communication unit that acquires video data from an image capturing device using the first communication path selected by the first path selection unit;
a processing unit that executes a first process using the video data acquired by the first communication unit;
a second communication unit that transmits the video data that has been subjected to the first processing by the processing unit to a data processing device via a second communication path. The relay device described in claim 1, characterized in that the first route selection unit selects the first communication route based on information specifying the data volume of the video data included in the information of the imaging device. The relay device described in claim 2, characterized in that the first path selection unit selects the first communication path based on information representing image quality as information defining the data volume of the video data. a second path selection unit that selects the second communication path based on the priority of each of a plurality of communication paths between the data processing device and the second communication path selection unit;
The relay device according to claim 1 , wherein the second communication unit transmits the video data using the second communication path selected by the second path selection unit. The relay device according to claim 1 , wherein the first route selection unit transmits to each of the image capturing devices an instruction to change the settings of the information of the image capturing devices used to select each of the first communication routes . when it is difficult to select the first communication path from among the plurality of communication paths based on the information of the image capturing device, the first path selection unit selects a third communication path other than the first communication path from among the plurality of communication paths based on the information of the image capturing device, and changes a setting of the image capturing device so that the image capturing device transmits video data via the third communication path;
The relay device according to claim 1 , wherein the first communication unit acquires video data from an image capturing device using the third communication path selected by the first path selection unit. The relay device
a first route selection step of selecting, for each of a plurality of photographing devices, a first communication route from among a plurality of communication routes between the photographing device and the respective photographing devices based on information about the photographing device, and when a plurality of photographing devices have selected the same specific communication route as the first communication route and the communication speed of the specific communication route falls below a threshold, reselecting, as the first communication route, a communication route having a lower transfer rate than the specific communication route for at least one of the photographing devices for which the specific communication route has been selected;
a first communication step of acquiring video data from an image capturing device using the first communication path selected by the first path selection step;
a processing step of executing a first process using the video data acquired by the first communication step;
a second communication step of transmitting the video data that has been subjected to the first processing by the processing step to a data processing device using a second communication path. a first route selection step of selecting, for each of a plurality of photographing devices, a first communication route from among a plurality of communication routes between the photographing device and the respective photographing devices based on information about the photographing device, and when a plurality of photographing devices have selected the same specific communication route as the first communication route and the communication speed of the specific communication route falls below a threshold, reselecting, as the first communication route, a communication route having a lower transfer rate than the specific communication route for at least one of the photographing devices for which the specific communication route has been selected;
a first communication step of acquiring video data from an image capturing device using the first communication path selected in the first path selection step;
a processing step of performing a first process using the video data acquired in the first communication step;
a second communication step of transmitting the video data that has been subjected to the first processing by the processing step to a data processing device via a second communication path. A relay system having a plurality of imaging devices, a relay device, and a data processing device,
the plurality of image capturing devices are each connected to the relay device via a plurality of communication paths;
The relay device
a first path selection unit that selects , for each of the plurality of image capture devices, a first communication path from among the plurality of communication paths based on information about the image capture device , and when there are a plurality of image capture devices that have selected the same specific communication path as the first communication path and the communication speed of the specific communication path falls below a threshold, reselects, as the first communication path, a communication path having a lower transfer rate than the specific communication path for at least one of the image capture devices for which the specific communication path has been selected ;
a first communication unit that acquires video data from an image capturing device using the first communication path selected by the first path selection unit;
a processing unit that performs a first process using the video data acquired by the first communication unit;
a second communication unit that transmits the video data that has been subjected to the first processing by the processing unit to the data processing device using a second communication path, and the data processing device performs second processing based on the video data that has been subjected to the first processing.