JP7634258B2 - Network scanning device, program for causing a computer to execute the program, computer-readable recording medium having the program recorded thereon, and data production method - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies January 9, 2021, Published on page 1 of the 2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC)

Article 30, paragraph 2 of the Patent Act applies January 14, 2021, Published in the IEICE Technical Report, NS2020-118, page 1

This invention relates to a network scanning device, a program for causing a computer to execute the program, a computer-readable recording medium on which the program is recorded, and a data production method.

As the fifth generation mobile communication (5G) system comes into practical use, many devices, such as mobile terminals, computers, sensors, and webcams, will be connected to the Internet. Such mobile terminals and computers connected to the Internet are called IoT (Internet of Things) devices.

In the IoT era, the number of IoT devices connected to communication networks is expected to increase rapidly. Among these, IoT devices that perform wireless communication are expected to be used illegally without sufficient security measures in order to increase the number of cases where they are exploited in cyber attacks such as distributed denial of service (DDoS) attacks.

To avoid such a situation, it is necessary to conduct network scans of IoT devices connected to communication networks and investigate whether security measures are in place for each IoT device.

The network scan described in Non-Patent Document 1 is known. The network scan described in Non-Patent Document 1 scans applications in the IPv4 space, and is performed by the following method.

First, a port scan is performed in the IPv4 space to check for responses. Then, for hosts that respond, a handshake is performed using various protocols (HTTP (Hypertext Transfer Protocol), HTTPS (Hypertext Transfer Protocol Secure), POP3 (Post Office Protocol version 3), etc.) to obtain messages. Keywords that are useful for vulnerability assessment are then extracted from the obtained messages. Metadata and tag information are then assigned to the keywords. The keywords with the assigned metadata and tag information are then registered in the database as structured data.

Z. Durumeric, et al., “A Search Engine Backed by Internet-Wide Scanning”, CCS15, October 12-16, 2015. Katsunari Yoshioka, Kosuke Murakami, Tsutomu Matsumoto, "Network Scanning Method for Detecting Malware-Infected Hosts and Automatic Generation of Detection Signatures," Transactions of the Information Processing Society of Japan, Vol. 51, No. 9, pp. 1633-1644 (Sep. 2010). G. F. Riley and T. R. Henderson, “The ns-3 Network Simulator,” In: K. Wehrle, M. Gunes M., and Gross J. (eds) Modeling and Tools for Network Simulation. Springer, Berlin, Heidelberg, 2010. 3GPP TS 36.814 v9.2.0-“A.2 System simulation scenarios”. B. Li, and L. Han, “Distance Weighted Cosine Similarity Measure for Text Classification,” In: Yin H. et al. (eds) Intelligent Data Engineering and Automated Learning - IDEAL 2013. IDEAL 2013. Lecture Notes in Computer Science, vol 8206. Springer, Berlin, Heidelberg. A. Heidarian and M. J. Dinneen, “A Hybrid Geometric Approach for Measuring Similarity Level Among Documents and Document Clustering,” 2016 IEEE Second International Conference on Big Data Computing Service and Applications (BigDataService), Oxford, 2016, pp. 142-151, DOI: 10.1109/BigDataService.2016.14. M. Mihajlovic and N. Xiong, “Finding the most similar textual documents using Case-Based Reasoning,” arXiv preprint arXiv:1911.00262, 2019. [Online]. https://arxiv.org/abs/1911.00262.

When a network scan is performed in a communications environment where the results of the network scan are unknown, misidentification of the network scan results may lead to erroneous inference of the cause of the failure of the network scan. This misidentification of the network scan results may occur due to the following factors:
(Cause 1) A similarity mismatch occurs because the magnitude, direction, and angle of a sample of a scan response, etc. obtained when a network scan is performed in a communication environment where the results of the network scan are unknown are very similar to a sample of a scan response, etc. obtained when a network scan is performed in a communication environment where the results of the network scan are known.
(Cause 2) The difference in size between samples of scan responses, etc. obtained when a network scan is performed in a communication environment where the results of the network scan are unknown is similar to the difference in size between samples of scan responses, etc. obtained when a network scan is performed in a communication environment where the results of the network scan are known.
(Cause 3) When compared with multiple known results of a network scan, the range of scan response delays obtained when performing a network scan in a communication environment where the results of the network scan are unknown overlap.
(Cause 4) The sample size of scan responses, etc. obtained when a network scan is performed in a communication environment where the results of the network scan are unknown is different from the sample size of scan responses, etc. obtained when a network scan is performed in a communication environment where the results of the network scan are known.

Therefore, according to an embodiment of the present invention, a network scanning device is provided that can reduce misidentification of the results of a network scan performed in a communication environment where the results of the network scan are unknown.

In addition, according to an embodiment of the present invention, a program is provided for causing a computer to execute a program to suppress misidentification of the results of a network scan performed in a communication environment where the results of the network scan are unknown.

Furthermore, according to an embodiment of the present invention, a computer-readable recording medium is provided that stores a program for causing a computer to execute a program to suppress misidentification of the results of a network scan performed in a communication environment where the results of the network scan are unknown.

Furthermore, according to an embodiment of the present invention, a data production method is provided for producing data used to suppress misidentification of the results of a network scan performed in a communication environment where the results of the network scan are unknown.

(Configuration 1)
According to an embodiment of the present invention, a network scanning device includes a scanning means, a calculating means, a detecting means, and a determining means, wherein the scanning means executes a network scan in a first communication environment in which the results of the network scan are unknown for M terminal devices (M is an integer equal to or greater than 2), and obtains a first scan response delay of the network scan for each of the M terminal devices. The calculation means, when the M first scan response delays are acquired by the scanning means, executes a first calculation process to calculate M' baseline scan response delays for each of a plurality of sets of M'(M' is an integer of 2 or more) second scan response delays obtained when results of a network scan executed in a second communication environment in which results of the network scan are known are a plurality of known estimated results, and to calculate M baseline scan response delays for the M first scan response delays, and a second calculation process to calculate similarity rates between the M' baseline scan response delays and the M baseline scan response delays for all of the plurality of sets of M' baseline scan response delays based on the plurality of sets of M' baseline scan response delays and the M baseline scan response delays calculated in the first calculation process. When the second calculation process is executed by the calculation means, the detection means detects the largest similarity rate among the plurality of similarity rates calculated in the second calculation process. When the detection means detects the highest similarity rate, the determination means determines that the unknown estimated result of the network scan when the M first scan response delays are acquired is a known estimated result corresponding to the M' second scan response delays having the highest similarity rate, and the baseline scan response delay is a scan response delay whose distribution is changed based on a lower limit and an upper limit so as to make it easier to obtain a similarity or difference in distribution between the M first scan response delays and the M' second scan response delays.

(Configuration 2)
In the first configuration, the network scanning device further includes an updating means for executing an aggregation process of aggregating the M′ second scan response delays and the M first scan response delays when the largest similarity rate is obtained, and updating the M′ second scan response delays when the largest similarity rate is obtained by the aggregated M′ second aggregated scan response delays in the aggregation process.

(Configuration 3)
In configuration 2, when there are multiple sets of M first scan response delays when the highest similarity rate is obtained, the update means selects, in an aggregation process, any set of M first scan response delays from the multiple sets of M first scan response delays, and aggregates the selected set of M first scan response delays with M' second scan response delays when the highest similarity rate is obtained.

(Configuration 4)
In configuration 2 or 3, in the aggregation process, the update means adds first scan response delays among the M first scan response delays that are not included in the M' second scan response delays when the highest similarity rate is obtained to the M' second scan response delays when the highest similarity rate is obtained, and aggregates the M' second scan response delays when the highest similarity rate is obtained and the M first scan response delays.

(Configuration 5)
In any of configurations 1 to 4, the calculation means, in the first calculation process, calculates a baseline scan response delay by subtracting a predetermined value from an average of the plurality of scan response delays and adding a predetermined value to the average, for all of the plurality of scan response delays to calculate a plurality of baseline scan response delays, and calculates a plurality of sets of M' baseline scan response delays and M baseline scan response delays by performing the process for all of the plurality of scan response delays. Then, the predetermined value is calculated based on a relative variation of the plurality of scan response delays.

(Configuration 6)
In configuration 5, in the first calculation process, the calculation means subtracts the inverse of the coefficient of variation in the distribution of the multiple scan response delays from the scan response delay, multiplies the subtraction result by the standard deviation of the multiple scan response delays, and divides the multiplication result by the number of the multiple scan response delays to calculate a predetermined value.

(Configuration 7)
In any of configurations 1 to 6, in the second calculation process, the calculation means calculates a first mean absolute deviation that is the mean absolute deviation of M baseline scan response delays, calculates a second mean absolute deviation that is the mean absolute deviation of M' baseline scan response delays for all of the multiple sets of M' baseline scan response delays to calculate a plurality of second mean absolute deviations, and calculates a similarity rate for all of the multiple second mean absolute deviations using the formula 1/(1+(Y-X) ² /XY), where X is the first mean absolute deviation and Y is any of the multiple second mean absolute deviations, to calculate a plurality of similarity rates.

(Configuration 8)
In any one of configurations 1 to 7, the network scanning device further includes a generating means, an algorithm updating means, and an estimating means. The generating means acquires, for all of the M' second scan response delays, an optimal communication characteristic that is the communication characteristic most suitable for estimating a known estimation result among a plurality of communication characteristics acquired together with one of the M' second scan response delays, and generates similar standard sample data consisting of the acquired M' optimal communication characteristics and the M' second aggregate scan response delays. The algorithm updating means trains an estimation algorithm that estimates a result of a network scan performed in the first communication environment based on the similar standard sample data, and updates the estimation algorithm using an evaluation index so as to approach the optimal algorithm. The estimation means estimates a result of a network scan performed in the first communication environment using the estimation algorithm updated by the algorithm updating means. The optimal algorithm includes an algorithm that estimates a result of a network scan such that the estimated result when the result of the network scan is estimated matches a known estimation result. The evaluation index is an index that evaluates the accuracy of an estimation result when the result of the network scan is estimated by the trained algorithm.

(Configuration 9)
Also, according to an embodiment of the present invention, the program includes a first step of a scanning means executing a network scan in a first communication environment in which the results of the network scan are unknown for M (M is an integer equal to or greater than 2) terminal devices, and acquiring a first scan response delay of the network scan for each of the M terminal devices;
a second step in which the calculation means, when M first scan response delays are acquired by the scanning means, executes a first calculation process in which the calculation means calculates M' baseline scan response delays for each of a plurality of sets of M'(M' is an integer of 2 or more) second scan response delays obtained when results of a network scan executed in a second communication environment in which results of the network scan are known are a plurality of known estimated results, and calculates M baseline scan response delays for the M first scan response delays; and a second calculation process in which the calculation means calculates similarity rates between the M' baseline scan response delays and the M baseline scan response delays for all of the plurality of sets of M' baseline scan response delays, based on the plurality of sets of M' baseline scan response delays and the M baseline scan response delays calculated in the first calculation process;
a third step of detecting a largest similarity rate among the plurality of similarity rates calculated in the second calculation process by the detection means when the second calculation process is executed by the calculation means;
and a fourth step in which the determining means determines, when the highest similarity rate is detected in the third step, that the unknown estimated result of the network scan when the M first scan response delays are acquired is a known estimated result corresponding to the M' second scan response delays having the highest similarity rate. The baseline scan response delay is a scan response delay whose distribution has been changed based on a lower limit and an upper limit so as to make it easier to obtain a similarity or difference in distribution between the M first scan response delays and the M' second scan response delays.

(Configuration 10)
In configuration 9, the updating means further causes the computer to execute a fifth step in which, when the highest similarity rate is detected in the third step, the updating means executes an aggregation process of aggregating M first scan response delays and M' second scan response delays when the highest similarity rate is obtained, and updates M' second scan response delays when the highest similarity rate is obtained by the M' second aggregated scan response delays aggregated in the aggregation process.

(Configuration 11)
In configuration 10, when there are multiple sets of M first scan response delays when the highest similarity rate is obtained, the update means selects, in an aggregation process, any set of M first scan response delays from the multiple sets of M first scan response delays, and aggregates the selected set of M first scan response delays with M' second scan response delays when the highest similarity rate is obtained.

(Configuration 12)
In configuration 10 or configuration 11, in the aggregation process, the update means adds first scan response delays among the M first scan response delays that are not included in the M' second scan response delays when the highest similarity rate is obtained to the M' second scan response delays when the highest similarity rate is obtained, and aggregates the M' second scan response delays when the highest similarity rate is obtained and the M first scan response delays.

(Configuration 13)
In any of configurations 9 to 12, the calculation means, in the first calculation process, performs a process of calculating a baseline scan response delay by subtracting a predetermined value from an average of the plurality of scan response delays and an addition result by adding a predetermined value to the average of the plurality of scan response delays, for all of the plurality of scan response delays to calculate a plurality of baseline scan response delays, and performs the process of calculating a plurality of baseline scan response delays by performing the process of calculating a plurality of sets of M' first scan response delays and all of the plurality of sets of M' second scan response delays to calculate a plurality of sets of M' baseline scan response delays and M baseline scan response delays, and the predetermined value is calculated based on a relative variation of the plurality of scan response delays.

(Configuration 14)
In configuration 13, in the first calculation process, the calculation means subtracts the inverse of the coefficient of variation in the distribution of the multiple scan response delays from the scan response delay, multiplies the subtraction result by the standard deviation of the multiple scan response delays, and divides the multiplication result by the number of the multiple scan response delays to calculate a predetermined value.

(Configuration 15)
In any of configurations 9 to 14, in the second calculation process, the calculation means calculates a first mean absolute deviation that is the mean absolute deviation of M baseline scan response delays, calculates a second mean absolute deviation that is the mean absolute deviation of M' baseline scan response delays for all of the multiple sets of M' baseline scan response delays to calculate a plurality of second mean absolute deviations, and calculates a similarity rate for all of the multiple second mean absolute deviations using the formula 1/(1+(Y-X) ² /XY), where X is the first mean absolute deviation and Y is any of the multiple second mean absolute deviations, to calculate a plurality of similarity rates.

(Configuration 16)
In any one of configurations 9 to 15, the program includes a sixth step in which the generating means acquires, for each of the M' second scan response delays, an optimal communication characteristic that is the communication characteristic most suitable for estimating a known estimation result among a plurality of communication characteristics acquired together with one of the M' second scan response delays, and generates similar standard sample data consisting of the acquired M' optimal communication characteristics and the M' second aggregate scan response delays;
a seventh step in which the algorithm update means trains an estimation algorithm for estimating the results of the network scan executed in the first communication environment based on the similar standard sample data, and updates the estimation algorithm using an evaluation index so as to approach an optimal algorithm;
and an eighth step in which the estimation means estimates the results of a network scan performed in the first communication environment using the estimation algorithm updated by the algorithm update means. The optimal algorithm is an algorithm that estimates the results of a network scan such that the estimated results of the network scan match a known estimated result. The evaluation index is an index that evaluates the accuracy of the estimated results of the network scan when the results of the network scan are estimated by the trained algorithm.

(Configuration 17)
Furthermore, according to an embodiment of the present invention, a recording medium is a computer-readable recording medium having the program according to any one of the ninth to sixteenth configurations recorded thereon.

(Configuration 18)
Further, according to an embodiment of the present invention, a data production method is a data production method for producing data used to estimate a result of a network scan, the data production method comprising:
A first step of executing a network scan in a first communication environment in which the results of the network scan are unknown for M (M is an integer equal to or greater than 2) terminal devices, and acquiring a first scan response delay of the network scan for each of the M terminal devices;
a second step of executing a first calculation process, when M first scan response delays are acquired in the first step, to calculate M' baseline scan response delays for each of multiple sets of M'(M' is an integer of 2 or more) second scan response delays obtained when results of a network scan executed in a second communication environment in which results of the network scan are known are multiple known estimated results, and to calculate M baseline scan response delays for the M first scan response delays; and a second calculation process, based on the multiple sets of M' baseline scan response delays and the M baseline scan response delays calculated in the first calculation process, to calculate similarity rates between the M' baseline scan response delays and the M baseline scan response delays for all of the multiple sets of M' baseline scan response delays to calculate multiple similarity rates.
When the second calculation process is executed in the second step, a third step of detecting the largest similarity rate among the multiple similarity rates calculated in the second calculation process;
a fourth step of executing an aggregation process when the highest similarity rate is detected in the third step, aggregating the M' second scan response delays and the M first scan response delays when the highest similarity rate is obtained, and updating the M' second scan response delays when the highest similarity rate is obtained by the M' second aggregated scan response delays aggregated in the aggregation process;
and a fifth step of acquiring, for all of the M' second scan response delays, an optimal communication characteristic that is the communication characteristic most suitable for estimating a known estimation result among the multiple communication characteristics acquired together with one of the M' second scan response delays updated in the fourth step, and generating similar standard sample data consisting of the acquired M' optimal communication characteristics and the M' second scan response delays. The baseline scan response delay is a scan response delay whose distribution has been changed based on a lower limit and an upper limit so as to make it easier to obtain a similarity or difference in distribution between the M first scan response delays and the M' second scan response delays.

(Configuration 19)
In configuration 18, in the aggregation process of the fourth step, among the M first scan response delays, first scan response delays that are not included in the M' second scan response delays when the highest similarity rate is obtained are added to the M' second scan response delays when the highest similarity rate is obtained, and the M' second scan response delays when the highest similarity rate is obtained and the M first scan response delays are aggregated.

(Configuration 20)
In configuration 18 or configuration 19, in the first calculation process of the second step, a process of calculating a baseline scan response delay by subtracting a predetermined value from an average of the multiple scan response delays and an addition result by adding a predetermined value to the average of the multiple scan response delays is performed for all of the multiple scan response delays to calculate multiple baseline scan response delays, and this process is performed for all of the multiple scan response delays to calculate M' baseline scan response delays and multiple sets of M' second scan response delays, and the predetermined value is calculated based on a relative variation of the multiple scan response delays.

(Configuration 21)
In configuration 20, in the first calculation process of the second step, the reciprocal of the coefficient of variation of the multiple scan response delays is subtracted from the scan response delay, the subtraction result is multiplied by the standard deviation of the multiple scan response delays, and the multiplication result is divided by the number of the multiple scan response delays to calculate a predetermined value.

(Configuration 22)
In any of configurations 18 to 21, in the second calculation process of the second step, a first mean absolute deviation that is the mean absolute deviation of M baseline scan response delays is calculated, and a second mean absolute deviation that is the mean absolute deviation of M' baseline scan response delays is calculated for all of the multiple sets of M' baseline scan response delays to calculate multiple second mean absolute deviations, and when the first mean absolute deviation is X and one of the multiple second mean absolute deviations is Y, a similarity rate is calculated for all of the multiple second mean absolute deviations using the formula 1/(1+(Y-X) ² /XY) to calculate multiple similarity rates.

(Configuration 23)
In any of configurations 18 to 22, the data production method includes a sixth step of training an estimation algorithm that estimates the results of the network scan using similar standard sample data;
a seventh step of estimating known results of the network scan using the estimation algorithm trained in the sixth step;
and an eighth step of determining whether the estimation result estimated in the seventh step matches a known estimation result,
The fifth step is executed when it is determined in the eighth step that the estimation result estimated in the seventh step matches a known estimation result.

This can reduce misinterpretation of the results of network scans performed in communication environments where the results of network scans are unknown.

1 is a schematic diagram of a communication system according to an embodiment of the present invention. FIG. 2 is a schematic diagram of the network scanning device shown in FIG. 1 . 3 is a conceptual diagram of a correspondence table TBL1 stored in the host information database shown in FIG. 2. 3 is a conceptual diagram of another correspondence table TBL2 stored in the host information database shown in FIG. 2. FIG. 4 is a conceptual diagram of yet another correspondence table TBL3 stored in the host information database shown in FIG. 3. FIG. 2 is a conceptual diagram of a scan schedule. FIG. 1 illustrates the time dependency of traffic. FIG. 1 illustrates an example of normalized scan response delay. A figure showing the score z _scr and baseline scan response delay b _RTT calculated for one terminal device. FIG. 13 illustrates an example of a baseline scan response delay b _RTT calculated by randomly selecting one scan response delay RTT. FIG. 13 is a diagram showing an example of a baseline scan response delay _bRTT calculated using all of a plurality of scan response delays RTT corresponding to a plurality of ports in one terminal device. FIG. 2 is a diagram showing a z-table. FIG. 13 shows baseline scan response latency _bRTT calculated using score _zβ . FIG. 1 illustrates an example of a baseline scan response delay _bRTT . FIG. 15 is a diagram showing the results of counting the number of samples for each value of the baseline scan response delay _bRTT shown in FIG. 14. FIG. 16 shows the distribution of baseline scan response delays _bRTT shown in FIG. 15. FIG. 15 is a diagram showing the results of changing some of the values of the baseline scan response delay _bRTT shown in FIG. 14. FIG. 18 shows the distribution of baseline scan response delays _bRTT shown in FIG. 17. FIG. 19 is a diagram showing the distribution of baseline scan response delays _bRTT when the scale of the horizontal axis in FIG. 18 is changed to "0.005" units. FIG. 13 shows the correspondence between the area of baseline scan response latency and the number of samples produced. FIG. 13 illustrates an exemplary distribution of probability density functions of baseline scan response delays. FIG. 13 illustrates another example of baseline scan response delay _bRTT . FIG. 23 shows the distribution of baseline scan response delays _bRTT shown in FIG. 22. FIG. 24 is a diagram showing the distribution of baseline scan response delays _bRTT when the scale of the horizontal axis in FIG. 23 is changed to "0.005" units. FIG. 13 is a diagram showing the relationship between the similarity rate b _sim and the mean absolute deviation W _known shown in equation (12). FIG. 13 is a diagram for explaining a method for detecting the maximum similarity rate b _{sim_max} . FIG. 13 is a diagram for explaining a method for aggregating two similar sets. FIG. 13 is a diagram for explaining a method of updating a known scan response delay {RTT _m' _known}_z (m'=1' to M', z=1 to 3). FIG. 13 is a diagram showing a similarity ratio b _sim calculated using three types of scores z _{scr — c} , z _{scr — 1} to z _{scr — R} , and z _β . FIG. 13 is a diagram for explaining a problem that occurs when determining a set S _G is a similar set. FIG. 13 shows the distribution of scan response delays RTT for sets S _E , S _F , and S _G and the baseline scan response delay b _RTT . FIG. 1 shows the similarity rate of two sets. _FIG . 13 is a diagram showing examples of scan response delays {RTT _m' — known}_ 3 (m'=1' to M') and {RTT _m _ unknown}_ 1 (m=1 to M) of sets _S E and S F . FIG. 13 is a diagram showing a set S _H obtained by aggregating sets S _E and S _F. 35 is a diagram showing an example of a scan response delay {RTT _m' _known}_3 (m'=1' to M') of the set S _E shown in FIG. 34 and an example of a scan response delay RTT of the set S _F that is not common between the sets S _E and S _F. FIG. FIG. 13 is a diagram showing the scan response delay {RTT _m _unknown}_2 (m=1 to M) of a set S _G. FIG. 13 shows baseline scan response latencies for sets S _G and S _H. FIG. 13 shows the distribution of scan response delays RTT of sets S _G and S _H and the baseline scan response delay b _RTT . 13 is a diagram showing the similarity rate b _sim between sets S _C , S _D , and S _H and set S _G. FIG. FIG. 1 illustrates a comparison of aggregating scan response delays with a method that does not aggregate scan response delays. FIG. 13 is a diagram illustrating a correlation coefficient of communication characteristics. FIG. 13 is a diagram illustrating another correlation coefficient of communication characteristics. FIG. 13 is a diagram showing still another correlation coefficient of communication characteristics. FIG. 13 is a diagram showing still another correlation coefficient of communication characteristics. FIG. 13 is a diagram showing an example of original data when selecting a communication characteristic CMbest. FIG. 13 is a diagram for explaining a method for selecting a communication characteristic CMbest. FIG. 11 is a diagram for explaining a method for calculating a score of a communication characteristic. FIG. 13 is a diagram showing scores of communication characteristics. FIG. 13 is a diagram showing the calculation results of rank weight R _k . FIG. 13 is a diagram showing a specific distribution of scan response delays. FIG. 13 is a diagram showing a specific distribution of scan response delays. FIG. 13 is a diagram showing a specific distribution of scan response delays. FIG. 13 shows the correspondence between PDF regions and the number of samples generated. FIG. 13 is a diagram showing an example distribution of a probability density function of a scan response delay. FIG. 13 illustrates another exemplary distribution of scan response delays. FIG. 1 shows data when training the estimation algorithm ALG. FIG. 1 is a conceptual diagram of sample data when training the estimation algorithm ALG. FIG. 13 is a conceptual diagram showing updating of virtual sample data for modeling. FIG. 1 is a conceptual diagram of training and evaluation of the estimation algorithm ALG. FIG. 59 is a diagram showing a specific example of the initial sample data D_sample_initial for modeling shown in FIG. 58. FIG. 57 is a diagram showing a specific example of the training set shown in FIG. 56. FIG. 57 is a diagram showing a specific example of the validation set shown in FIG. 56. FIG. 57 is a diagram showing a specific example of the test data shown in FIG. 56. FIG. 1 is a conceptual diagram of a confusion matrix. FIG. 13 is a diagram for explaining a method for generating a confusion matrix. 10 is a flowchart illustrating a network scan operation. 67 is a first flowchart for explaining the detailed operation of step S10 in FIG. 66. 67 is a second flowchart for explaining the detailed operation of step S10 of FIG. 66. 67 is a first flowchart for explaining the detailed operation of step S11 in FIG. 66. FIG. 67 is a second flowchart for explaining the detailed operation of step S11 in FIG. 66. 67 is a third flowchart for explaining the detailed operation of step S11 in FIG. 66. 70 is a flowchart for explaining detailed operations of step S134 in FIG. 69. 71 is a first flowchart for explaining a detailed operation of step S147 in FIG. 70. FIG. 71 is a second flowchart for explaining the detailed operation of step S147 in FIG. 70. 71 is a third flowchart for explaining the detailed operation of step S147 in FIG. 70. 74 is a flowchart for explaining detailed operations of step S1472 in FIG. 73. 74 is a flowchart for explaining detailed operations of step S1476 in FIG. 73. 71 is a flowchart for explaining detailed operations of step S149 in FIG. 70. 71 is a flowchart for explaining a detailed operation of step S149 in FIG. 70 when the Matthews correlation coefficient MCC is calculated in step S149. 67 is a first flowchart for explaining the detailed operation of step S13 of FIG. 66. FIG. 67 is a second flowchart for explaining the detailed operation of step S13 in FIG. 66. 67 is a first flowchart for explaining the detailed operation of step S14 in FIG. 66. 67 is a second flowchart for explaining the detailed operation of step S14 in FIG. 66. FIG. 13 is a diagram illustrating an example of a confusion matrix. FIG. 3 is a schematic diagram illustrating the operation of the network scanning device illustrated in FIG. 2 . 11 is a flowchart for explaining a method for producing standard sample data D_similarity. FIG. 13 is a diagram showing the relationship between the similarity rate b _sim and the results of a network scan. FIG. 13 is a diagram showing the relationship between the similarity rate b _sim and the number of ports to be scanned. FIG. 13 is a diagram showing a comparison of the similarity rate of the present method and the similarity rate of a conventional method. FIG. 1 illustrates the accuracy of estimating the results of a network scan. FIG. 13 is a diagram showing simulation results of D-score. FIG. 13 illustrates the probability density function of the scan response delay RTT _m .

The embodiment of the present invention will be described in detail with reference to the drawings. Note that the same or corresponding parts in the drawings are given the same reference numerals and their description will not be repeated.

Figure 1 is a schematic diagram of a communication system according to an embodiment of the present invention. Referring to Figure 1, a communication system 10 according to an embodiment of the present invention includes a network scanning device 1, base stations BS1 and BS2, an Internet provider ISP (Internet Service Provider), and terminal devices TM1 to TM18.

The network scanning device 1, base stations BS1 and BS2, the Internet provider ISP, and terminal devices TM1 to TM18 are placed in the communication space.

Base station BS1 is a base station of wireless communication network CNW1, base station BS2 is a base station of wireless communication network CNW2, and the Internet provider ISP is a communications carrier of wired communication network CNW3.

The wireless communication networks CNW1 and CNW2 are communication networks that perform wireless communication according to mutually different wireless communication standards.

In this embodiment of the invention, each of the terminal devices TM1 to TM13 is a wireless terminal, and each of the terminal devices TM14 to TM18 is a wired terminal.

The terminal devices TM1 to TM6 are located outside the wireless communication network CNW2 and inside the wireless communication network CNW1.

The terminal device TM7 is located within the overlapping area of the wireless communication networks CNW1 and CNW2.

Terminal devices TM8 to TM13 are located outside wireless communication network CNW1 and inside wireless communication network CNW2.

Terminal devices TM14 to TM18 are placed within the wired communications network CNW3 and are connected to the Internet provider ISP via wires.

Each of the terminal devices TM1 to TM13 is equipped with two wireless communication modules capable of wireless communication with base stations BS1 and BS2. Therefore, each of the terminal devices TM1 to TM13 is capable of wireless communication with base station BS1 or base station BS2.

Terminal devices TM1 to TM6 communicate wirelessly with base station BS1, terminal device TM7 communicates wirelessly with base station BS1 or base station BS2, and terminal devices TM8 to TM13 communicate wirelessly with base station BS2.

Terminal devices TM14 to TM18 communicate via wired connection via an Internet provider (ISP).

The network scanning device 1 performs network scanning on the terminal devices TM1 to TM18 using the method described below.

Figure 2 is a schematic diagram of the network scanning device 1 shown in Figure 1. Referring to Figure 2, the network scanning device 1 includes a network scanner 11, a scan analysis manager 12, a host information database 13, an update means 14, an extraction means 15, a calculation means 16, an estimation means 17, and a scan scheduler 18.

The network scanner 11 receives scan timing from the scan scheduler 18, and based on the received scan timing, performs a network scan on the ports of the terminal devices connected to the network 20 to be scanned, for example, using the well-known Nmap (see non-patent document 2).

Then, the network scanner 11 receives a scan response of the network scan from a terminal device connected to the network 20, and outputs the received scan response to the user (comprising network security, etc.) 30 and the scan analysis manager 12.

In addition, when a network scan is performed without monitoring background traffic, the network scanner 11 measures the scan response delay RTT _m (m is an integer satisfying 1≦m≦M, M is the total number of terminal devices TM1 to TM18 and is an integer of 2 or more), the packet loss rate PLR _m , the maximum scan delay DLmax _m and the minimum scan delay DLmin _m , and outputs the measured scan response delay RTT _m , the packet loss rate PLR _m , the maximum scan delay DLmax _m and the minimum scan delay DLmin _m to the scan analysis manager 12. That is, when a network scan is performed without monitoring background traffic, the network scanner 11 measures communication characteristics CM1 = [scan response delay RTT _m / packet loss rate PLR _m / maximum scan delay DLmax _m / minimum scan delay DLmin _m ] consisting of scan response delay RTT _m , packet loss rate PLR _m , maximum scan delay DLmax _m and minimum scan delay DLmin _m , and outputs the measured communication characteristics CM1 = [scan response delay RTT _m / packet loss rate PLR _m / maximum scan delay DLmax _m / minimum scan delay DLmin _m ] to the scan analysis manager 12.

Furthermore, when a network scan is performed by monitoring background traffic, the network scanner 11 measures the scan response delay RTT _m , packet loss rate PLR _m , delay DL _m , jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m , and outputs the measured scan response delay RTT _m , packet loss rate PLR _m , delay DL _m , jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m to the scan analysis manager 12. That is, when a network scan is performed by monitoring background traffic, the network scanner 11 measures communication characteristics CM2 = [scan response delay RTT _m / packet loss rate PLR _m / delay DL _m / jitter JT _m / background maximum delay BGDLmax _m / background minimum delay BGDLmin _m ] consisting of scan response delay RTT _m , packet loss rate PLR _m , delay DL _m , jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m , and outputs the measured communication characteristics CM2 = [scan response delay RTT _m / packet loss rate PLR _m / delay DL _m / jitter JT _m / background maximum delay BGDLmax _m / background minimum delay BGDLmin _m ] to the scan analysis manager 12.

The scan analysis manager 12 receives a scan response from the network scanner 11 and receives a communication log during the network scan. The scan analysis manager 12 also receives communication characteristics CM1 or CM2 from the network scanner 11.

Then, when the scan analysis manager 12 receives the scan response and the communication characteristics CM1, it generates a scan result SC1 that associates the address of the terminal device to be scanned, the scan response, and the communication characteristics CM1 based on the scan response and the communication characteristics CM1, and creates or updates the host information database 13 by storing the generated scan result SC1 in the host information database 13.

In addition, when the scan analysis manager 12 receives the scan response and communication characteristics CM2, it generates a scan result SC2 that associates the address of the terminal device to be scanned, the scan response, and the communication characteristics CM2 based on the scan response and communication characteristics CM2, and creates or updates the host information database 13 by storing the generated scan result SC2 in the host information database 13.

The host information database 13 prestores in association with each other the scan response delay {RTT _{m' _known}_1 (m' = 1' to M') acquired when a network scan is performed in a communication environment where the result of the network scan is "normal" and the communication characteristic {CMbest m '} _known}_1 (m' = 1' to M') most suitable for estimating that the result of the network scan is "normal". Here, m' is an integer that satisfies 1≦m'≦M', and M' is the total number of terminal devices that are targets of a network scan performed in a communication environment where the result of the network scan is known. Furthermore, m' may be the same as m or may be different from m.

In addition, the host information database 13 pre-stores in association with the scan response delay {RTT _{m' _known}_2 (m' = 1' to M') acquired when a network scan is performed in a communication environment where the network scan fails due to "insufficient communication quality", and the communication characteristic {CMbest m '} _known}_2 (m' = 1' to M') that is most suitable for inferring that the network scan has failed due to "insufficient communication quality".

Furthermore, the host information database 13 pre-stores in association with the scan response delay {RTT _{m' _known}_3 (m' = 1' to M') acquired when a network scan is executed in a communication environment where the network scan result is unsuccessful due to "network congestion" and the communication characteristic {CMbest m '} _known}_3 (m' = 1' to M') that is most suitable for estimating that the network scan result is unsuccessful due to "network congestion".

Furthermore, the host information database 13 stores the scan result SC1 or the scan result SC2.

The update means 14 reads out from the host information database 13 the scan response delay {RTT m _known} (m = 1 to M) obtained when the network scanner 11 performs a network scan in a communication environment where the results of the network scan are unknown, based on the communication characteristics CM1 (or communication characteristics CM2) stored in the host information database 13, and also reads out from the host information database 13 the scan response delay {RTT _{m '} _known} _1 (m' = 1' to M'), the scan response delay {RTT _m' _known} _2 (m' = 1' to M') and the scan response delay {RTT _m' _known} _3 (m' = 1' to M').

Then, the updating means 14 outputs the scan response delay {RTT _m _known} (m = 1 to M), the scan response delay {RTT _m' _known} _1 (m' = 1 to M'), the scan response delay {RTT _m' _known} _2 (m' = 1 to M') and the scan response delay {RTT _m' _known} _3 (m' = 1 to M') to the calculation means 16.

Then, the updating means 14 receives three similarity rates b sim1 to b sim3 between the scan response delay {RTT _m _known} (m = 1 to M) and the scan response delay {RTT _{m' _known} }_1 (m' = 1' to M'), the scan response delay {RTT m' _known}_2 (m' = 1' to M') and the scan response delay _{ RTT _m' _known}_3 (m' = 1' to M') from the calculation means 16, and detects the largest similarity rate b _{sim_max} from the three similarity rates _{b sim1 to} b _sim3 .

Then, the updating means 14 _aggregates the scan response delay {RTT _m' _known}_similarity (m'=1' to M') (any of scan response delay {RTT m _' _known}_1 (m'=1' to M'), scan response delay {RTT _m' _known}_2 (m'=1' to M'), and scan response delay {RTT _m' _known}_3 (m'=1' to M')) when the largest similarity rate b sim_max is obtained, and the scan response delay {RTT _m _unknown} (m=1 to M), and updates the correspondence table TBL3 stored in the host information database 13 with the aggregated scan response delay.

The extraction means 15 executes an extraction process to extract communication characteristics {CMbest _m _unknown} (m = 1 to M) most suitable for estimating the results of a network scan by a method described later based on the communication characteristics CM1 (or communication characteristics CM2) stored in the host information database 13. Then, the extraction means 15 generates sample data D_unknown consisting of scan response delays {RTT _m _unknown} (m = 1 to M) and communication characteristics {CMbest _m _unknown} (m = 1 to M), and outputs the generated sample data D_unknown to the calculation means 16 and the estimation means 17.

In addition, the extraction means 15 detects the scan response delay {RTT _m' _known}_1 (m' = 1' to M') and communication characteristics {CMbest _m ' _known}_1 (m' = 1' to M') acquired in a communication environment where the result of the network scan is "normal" from the updated correspondence table TBL3 of the host information database 13, generates standard sample data D_standard_1 consisting of the detected scan response delay {RTT _m' _known}_1 (m' = 1' to M') and communication characteristics {CMbest _m' _known}_1 (m' = 1' to M'), and outputs the generated standard sample data D_standard_1 to the calculation means 16.

Furthermore, the extraction means 15 detects the scan response delay {RTT _m' _known}_2 (m' = 1' to M') and communication characteristics {CMbest _m ' _known}_2 (m' = 1' to M') acquired in a communication environment where the result of the network scan is "insufficient communication quality" from the updated correspondence table TBL3 of the host information database 13, generates standard sample data D_standard_2 consisting of the detected scan response delay {RTT _m' _known}_2 (m' = 1' to M') and communication characteristics {CMbest _m' _known}_2 (m' = 1' to M'), and outputs the generated standard sample data D_standard_2 to the calculation means 16.

Furthermore, the extraction means 15 detects the scan response delay {RTT _m' _known}_3 (m' = 1' to M') and communication characteristics {CMbest _m ' _known}_3 (m' = 1' to M') acquired in a communication environment where the result of the network scan is "network congestion" from the updated correspondence table TBL3 of the host information database 13, generates standard sample data D_standard_3 consisting of the detected scan response delay {RTT _m' _known}_3 (m' = 1' to M') and communication characteristics {CMbest _m' _known}_3 (m' = 1' to M'), and outputs the generated standard sample data D_standard_3 to the calculation means 16.

Furthermore, the extraction means 15 receives from the calculation means 16 three similarities P S1 , P S2 , and P S3 between each of the scan response delays {RTT _{m' _known} }_1 (m' = 1' to M') to {RTT _m' _known}_3 (m' = 1' to M' ₎ included in the standard sample data _{D_standard_1} to _{D_standard_3} and the scan response delay RTT m _known included in the sample data D_unknown, and calculates the scan response delay { _{RTT m' _known }} _similarity (m' = 1' to M') (= the scan response delay {RTT _m' _known}_1 (m'=1' to M') to {RTT _m' _known}_3 (m'=1' to M') are detected. Then, extraction means 15 outputs to estimation means 17 standard sample data D_similarity consisting of the scan response delay {RTT _m' _known}_similarity (m' = 1' to M') and the communication characteristic {CMbest _{m '} _known} similarity (= any of communication characteristics {CMbest _m' _known}_1 (m' = 1' to M') to {CMbest _m' _known}_3 (m' = 1' to M')) corresponding to the scan response delay {RTT m' _known}_similarity (m' = 1' to M').

The calculation means 16 receives the scan response delay {RTT _m _known} (m=1 to M), the scan response delay {RTT _m' _known} _1 (m'=1 to M'), the scan response delay {RTT _m' _known} _2 (m'=1 to M') and the scan response delay {RTT _m' _known} _3 (m'=1 to M') from the update means 14. Then, calculation means 16 calculates three similarity rates b sim1 to b sim3 between the scan response delay {RTT _m _known} (m = 1 to M) and the scan response delay {RTT _m' _known}_1 (m' = 1' to M'), the scan response delay {RTT _m' _known}_2 (m' = 1' to M') and the scan response delay {RTT _m' _known}_3 (m' = 1' to M _' ) using a method described later, and outputs the three calculated similarity rates _{b sim1 to} b _sim3 to update means 14.

The calculation means 16 also receives the sample data D_unknown and the standard sample data D_standard_1 to D_standard_3 from the extraction means 15. The calculation means 16 then calculates three similarities P S1 , P S2 , and P S3 between each of the scan response delays {RTT _m' _known}_1 (m' = 1' to M') to {RTT m _' _known}_3 (m' = 1' to M') included in the standard sample data D_standard_1 to D_standard_3 and the scan response delay RTT _m _known (m = 1 to M) included in the sample data D_unknown by _{a method described later, and outputs the three calculated similarities P S1 , P S2 , and P S3} to _the extraction means 15.

Furthermore, the calculation means 16 receives from the estimation means 17 the estimation results (confusion matrices CFM, nCFM, tCFM) obtained by estimating the results of the network scan using the estimation algorithm ALG. Then, the calculation means 16 calculates an evaluation index EVD representing the accuracy of the estimation result based on the confusion matrix CFM using a method described below, calculates an evaluation index nEVD representing the accuracy of the estimation result based on the confusion matrix nCFM, and calculates an evaluation index tEVD representing the accuracy of the estimation result based on the confusion matrix tCFM. Then, the calculation means 16 outputs the evaluation indexes EVD, nEVD, and tEVD to the estimation means 17.

The estimation means 17 combines the standard sample data D_standard_1 to D_standard_3 to generate initial sample data D_sample_initial for modeling.

The estimation means 17 also receives standard sample data D_similarity (consisting of scan response delay {RTT _m' _known}_similarity (m'=1' to M') and communication characteristic {CMbest _m' _known}_similarity (m'=1' to M')) from the extraction means 15. Then, the estimation means 17 divides the standard sample data D_similarity into training data and test data, and divides the training data into a training set and a validation set. Then, the estimation means 17 generates a plurality of tentative sample data D_sample_tentative based on the training set and the initial sample data D_sample_initial by a method described later.

Then, the estimation means 17 trains the estimation algorithm ALG using the initial sample data D_sample_initial, estimates the results of the network scan by the estimation algorithm ALG (the trained estimation algorithm ALG) using the validation set, and generates a confusion matrix CFM based on the estimated results of the network scan and outputs it to the calculation means 16.

Next, the estimation means 17 trains the estimation algorithm ALG using the tentative sample data D_sample_tentative, estimates the results of the network scan by the estimation algorithm ALG (the trained estimation algorithm ALG) using the validation set, and generates a confusion matrix nCFM based on the estimated results of the network scan and outputs it to the calculation means 16.

The estimation means 17 outputs the confusion matrix CFM to the calculation means 16, and then receives the evaluation index EVD from the calculation means 16, and outputs the confusion matrix nCFM to the calculation means 16, and then receives the evaluation index nEVD from the calculation means 16. Then, when the evaluation index nEVD is greater than the evaluation index EVD, the estimation means 17 sets the evaluation index nEVD to the evaluation index EVD. On the other hand, when the evaluation index nEVD is not greater than the evaluation index EVD, the estimation means 17 does not set the evaluation index nEVD to the evaluation index EVD.

The estimation means 17 performs a comparison between the evaluation index nEVD and the evaluation index EVD for all of the multiple tentative sample data D_sample_tentative, and replaces the variables of the estimation algorithm ALG with the tentative sample data D_sample_tentative when the largest evaluation index nEVD is obtained.

Then, the estimation means 17 estimates the results of the network scan by the estimation algorithm ALG (the estimation algorithm ALG with the variables replaced) using the test data, generates a confusion matrix tCFM based on the estimated results of the network scan, and outputs it to the calculation means 16. Then, the estimation means 17 receives the evaluation index tEVD from the calculation means 16, and when the evaluation index tEVD is equal to or greater than the evaluation index EVD, it approves the estimation algorithm ALG with the variables replaced and the tentative sample data D_sample_tentative when the largest evaluation index nEVD is obtained.

In this way, the estimation means 17 trains the estimation algorithm ALG and evaluates the accuracy of the estimation result when the network scan result is estimated using the trained estimation algorithm ALG.

Furthermore, the estimation means 17 estimates the results of the network scan using the estimation algorithm ALG in the following manner: The estimation means 17 classifies the terminal devices having the communication characteristics CMbest _m _unknown (m=1 to M) extracted by the extraction means 15 into a group CLS ₁ in which the network scan is normal, and groups CLS ₂ and CLS ₃ in which the network scan has failed due to a cause attributable to the communication channel with the terminal device.

The estimation means 17 then calculates a D-score D_2 for distinguishing between groups CLS ₁ and CLS ₃ , or a D-score D_3 for distinguishing between groups CLS ₂ and CLS ₃ , and estimates that the network scan is normal or unsuccessful based on the calculated D-score D_2 or D-score D_3, and when it estimates that the network scan is unsuccessful, executes an estimation process for estimating the cause of the failure of the network scan. The estimation means 17 then outputs the estimation result estimated in the estimation process to the scan scheduler 18.

The scan scheduler 18 reads the scan results from the host information database 13 and receives scan requirements from the user 30. The scan scheduler 18 also holds in advance the global IP addresses and domain names of all terminal devices present in the area (e.g., Japan) where the network scan is to be performed.

Then, based on the scan requirements and the scan results, the scan scheduler 18 generates a scan schedule SCH_scan that associates the scan timings of the network scan with the terminal devices that are the targets of the network scan at each scan timing, using a method described below. The scan scheduler 18 then outputs the scan schedule SCH_scan to the network scanner 11 and controls the network scanner 11 to execute a network scan in accordance with the scan schedule SCH_scan.

In addition, when the scan scheduler 18 receives the estimation result from the estimation means 17, it reschedules the scan schedule based on the cause of the network scan failure in the estimation result, outputs the rescheduled scan schedule RESCH_scan to the network scanner 11, and controls the network scanner 11 to execute the network scan in accordance with the scan schedule RESCH_scan.

Figure 3 is a conceptual diagram of the correspondence table TBL1 stored in the host information database 13 shown in Figure 2. Note that the correspondence table TBL1 shown in Figure 3 is a correspondence table when a network scan is performed without monitoring background traffic.

Referring to FIG. 3, correspondence table TBL1 includes time t, an IP address Add _{b_v} (v=1 to V, where V is the total number of base stations), an IP address Add _m , a domain name DN _m , a port number PN _r (r is a positive integer), a scan response SR _m , a scan response delay RTT _m , a packet loss rate PLR _m , a scan maximum delay DLmax _m , and a scan minimum delay DLmin _m .

Time t is expressed as YYYY/MM/DD/hh/mm/ss (year/month/day/hour/minute/second). Time t is a time within one scan timing having a certain time length, which will be described later, and represents the time when the test input in the network scan is sent to the terminal device that is the target of the network scan.

The IP address Add _{b_v} is the address of the base station, and the IP address Add _m is the global IP address of the terminal device that is the target of the network scan.

The domain name DN _m is converted based on the IP address Add _m using a PTR (Pointer record).

The port number _PNr is the number of the port that each terminal device uses for communication.

The scan response _SRm is set to "1" when a scan response is returned, and is set to "0" when no scan response is returned.

The scan response delay RTT _m is the time from the time the test input is transmitted until the scan response SR _m is returned.

The packet loss rate PLR _m is the rate at which packets fail to be transmitted and received when a packet consisting of a test input is transmitted and a scan response SR _m is received.

The scan maximum delay DLmax _m is the maximum value of the scan response delay RTT _m , and the scan minimum delay DLmin _m is the minimum value of the scan response delay RTT _m .

The scan response SR _m and the scan response delay RTT _m are associated with the port number PN _r . The packet loss rate PLR _m , the scan maximum delay DLmax _m and the scan minimum delay DLmin _m are associated with the IP address Add _m of the terminal device that is the target of the network scan.

Furthermore, addresses Add ₁ to Add ₃ are associated with address Add _{b_1} , ..., addresses Add _M-1 to Add _M are associated with address Add _{b_V} . As a result, it is possible to know the terminal devices belonging to each base station.

Figure 4 is a conceptual diagram of another correspondence table TBL2 stored in the host information database 13 shown in Figure 2. Note that the correspondence table TBL2 shown in Figure 4 is a correspondence table used when a network scan is performed by monitoring background traffic.

4, correspondence table TBL2 is the same as correspondence table TBL1, except that the scan maximum delay DLmax _m and scan minimum delay DLmin _m of correspondence table TBL1 shown in FIG. 3 are replaced with delay DL _m , jitter JT _m , background maximum delay BGDLmax _m and background minimum delay BGDLmin _m .

The delay DL _m , the jitter JT _m , the background maximum delay BGDLmax _m and the background minimum delay BGDLmin _m are associated with the IP address Add _m of each terminal device.

The delay DL _m is the delay from the standard time in one-way communication from the terminal device.

Jitter JT _m is the fluctuation in the transmission time of a packet in one-way communication from a terminal device.

The background maximum delay BGDLmax _m is the maximum value of the transmission delay in one-way communication from the terminal device, and the background minimum delay BGDLmin _m is the minimum value of the transmission delay in one-way communication from the terminal device.

In the correspondence table TBL2, the scan response SR _m and the scan response delay RTT _m are associated with the port number PN _r of each terminal device.

Figure 5 is a conceptual diagram of yet another correspondence table TBL3 stored in the host information database 13 shown in Figure 3.

5, correspondence table TBL3 includes a known estimation result, a time t', an IP address Add _{b'_w} (w=1 to W, where W is the total number of base stations), an IP address Add _m' , a scan response delay RTT _m' _known, and a communication characteristic CMbest _m' _known. The known estimation result, the time t', the IP address Add _{b'_w} , the IP address Add _m' , the scan response delay RTT _m' _known, and the communication characteristic CMbest _m' _known are associated with each other.

Known estimated results from network scans include "normal," "poor communication quality," and "network congestion."

The time t' has the same format as the time t described in Fig. 3. The IP address Add _{b'_w} is the address of the base station. The IP address Add _m' is the global IP address of the terminal device that is the target of the network scan. The IP addresses Add _1' to Add _3' are associated with the IP address Add _{b'_1} of the base station, and similarly, the IP address Add _M' is associated with the IP address Add _{b'_W} of the base station.

The scan response delay {RTT _1' _known}_1, the scan response delay {RTT _2' _known}_1, the scan response delay {RTT _3' _known}_1, ..., and the scan response delay {RTT _M' _known}_1 are respectively associated with IP address Add _1' , IP address Add _2' , IP address Add _3' , ..., IP address Add _M' , and are scan response delays RTTs obtained in a communication environment where the result of a network scan is "normal".

Communication characteristic {CMbest _1' _known}_1, communication characteristic {CMbest _2' _known}_1, communication characteristic {CMbest _3' _known}_1, ..., and communication characteristic {CMbest _M' _known}_1 are communication characteristics CMbest respectively associated with IP address Add _1' , IP address Add _2' , IP address Add _3' , ..., IP address Add _M' , and are obtained in a communication environment where the result of a network scan is "normal".

The scan response delay {RTT _1' _known}_2, the scan response delay {RTT _2' _known}_2, the scan response delay {RTT _3' _known}_2, ..., and the scan response delay {RTT _M' _known}_2 are respectively associated with IP address Add _1' , IP address Add _2' , IP address Add _3' , ..., IP address Add _M' , and are scan response delays RTTs obtained in a communication environment where the result of a network scan is "insufficient communication quality."

The communication characteristic {CMbest _1' _known}_2, the communication characteristic {CMbest _2' _known}_2, the communication characteristic {CMbest _3' _known}_2, ..., and the communication characteristic {CMbest _M' _known}_2 are communication characteristics CMbest that are associated with IP address Add _1' , IP address Add _2' , IP address Add _3' , ..., IP address Add _M' , respectively, and are acquired in a communication environment where the result of a network scan is "insufficient communication quality."

The scan response delay {RTT _1' _known}_3, the scan response delay {RTT _2' _known}_3, the scan response delay {RTT _3' _known}_3, ..., and the scan response delay {RTT _M' _known}_3 are scan response delays RTTs associated with IP address Add _1' , IP address Add _2' , IP address Add ₃ ', ..., IP address Add _M' , respectively, and obtained in a communication environment where the result of a network scan is "network congestion."

The communication characteristic {CMbest _1' _known}_3, the communication characteristic {CMbest _2' _known}_3, the communication characteristic {CMbest _3' _known}_3, ..., and the communication characteristic {CMbest _M' _known}_3 are communication characteristics CMbest that are associated with IP address Add _1' , IP address Add _2' , IP address Add _3' , ..., IP address Add _M' , respectively, and are acquired in a communication environment where the result of a network scan is "network congestion".

The scan response delays {RTT _{1 '} _known}_1 to {RTT _{M'_known} }_1 and the communication characteristics {CMbest _{1 '} _known}_1 to {CMbest _{M'_known} }_1 constitute standard sample data D_standard_1.

Furthermore, the scan response delays {RTT _{1 '} _known}_2 to {RTT _{M'_known} }_2 and the communication characteristics {CMbest _{1 '} _known}_2 to {CMbest _{M'_known} }_2 constitute standard sample data D_standard_2.

Furthermore, the scan response delays {RTT _{1 '} _known}_3 to {RTT _{M'_known} }_3 and the communication characteristics {CMbest _{1 '} _known}_3 to {CMbest _{M'_known} }_3 constitute standard sample data D_standard_3.

In addition, each of the scan response delays {RTT _1' _known}_1 to {RTT _M' _known}_1 shown in the correspondence table TBL3 actually exists in a plurality of pieces corresponding to a plurality of ports in one IP address Add _m' . Also, each of the communication characteristics {CMbest _1' _known}_1 to {CMbest _M' _known}_1 actually exists in a plurality of pieces corresponding to a plurality of ports in one IP address Add _m' . The same is true for the scan response delays {RTT _{1 '} _known}_2 to {RTT _{M'_known} }_2; {RTT _{1 '} _known}_3 to {RTT _{M'_known} }_3 and the communication characteristics {CMbest _{1 '} _known}_2 to {CMbest _{M'_known} }_2; {CMbest _{1 '} _known}_3 to {CMbest _{M'_known} }_3.

Further, the “1” of the scan response delays {RTT _1′ _known}_1 to {RTT _M′ _known}_1 and the “1” of the communication characteristics {CMbest _1′ _known}_1 to {CMbest _M′ _known}_1 represent an identification code D_code=1 assigned to “normal”, which is a known estimated result of the network scan. The “2” of the scan response delays {RTT _1′ _known}_2 to {RTT _M′ _known}_2 and the “2” of the communication characteristics {CMbest _1′ _known}_2 to {CMbest _M′ _known}_2 represent an identification code D_code=2 assigned to “insufficient communication quality”, which is a known estimated result of the network scan. The "3" in {RTT _{1'_known} }_3 to {RTT _{M'_known} }_3 and the "3" in {CMbest _{1'_known} }_3 to {CMbest _{M'_known} }_3 represent the identification code D_code=3 assigned to "network is congested", which is a known estimated result of the network scan.

In this embodiment of the invention, when a network scan is performed without monitoring background traffic, correspondence tables TBL1 and TBL3 are stored in the host information database 13, and when a network scan is performed while monitoring background traffic, correspondence tables TBL2 and TBL3 are stored in the host information database 13. Correspondence table TBL3 is stored in the host information database 13 before the network scanner 11 starts a network scan in a communication environment where the results of the network scan are unknown.

Fig. 6 is a conceptual diagram of a scan schedule. Referring to Fig. 6(a), the scan schedule SCH_scan has a total scan time length T _S _total, and includes T (T is an integer equal to or greater than 1) scan timings 1 _S to T _S.

The total scan time length T _S _total is set to, for example, one day, and each of scan timings 1 _S to _TS has a time length of, for example, one hour. The scan timings t _S (t _S =1 _S to _TS ) include a start time t _start , an end time t _end , and target addresses Add ₁ to Add _M.

The start time t _start represents the time when the network scan starts at the scan timing t _S , and the end time t _end represents the time when the network scan ends at the scan timing t _S. Therefore, the network scanner 11 ends the network scan when the end time t _end arrives at the scan timing t _S.

Each of the target addresses _{Add 1} to Add _M is a global IP address, and represents _M terminal devices that are targets of the network scan.

Each of the scan timings 1 _S to (t-1) _S and (t+1) _S to T _S other than the scan timing t _S has the same configuration as the scan timing t _S shown in FIG. 6( a ).

The scan timings _1S to _TS may have the same time length (=t _end -t _start ) or may have different time lengths (=t _end -t _start ). Also, the scan timings _1S to _TS may have the same number of IP addresses or may have different numbers of IP addresses.

When a network scan is executed for the first time, the scan timing 1 _S to T _S shown in FIG. 6(a) is used.

Furthermore, when a network scan is performed during times when communication is not busy, the scan timings 1 _'S to T _'S (T' is an integer equal to or greater than 1) shown in Fig. 6(b) are used. In Fig. 6(b), the scan timings 1 _'S to T _'S are set to be spaced apart in time, because multiple time periods during which communication is not busy are never adjacent to each other.

The configuration of the scan timing t' _S (t' _S =1' _S to T' _S ) is the same as the configuration of the scan timing t _S described above.

Moreover, the scan timings t _S and t' _S do not mean a single time, but are a concept having a predetermined time length from the start time t _start to the end time t _end .

A method for determining the scan timing t _'S will be described. Fig. 7 is a diagram showing the time dependency of traffic. In Fig. 7, the vertical axis represents traffic and the horizontal axis represents time. Furthermore, curve k1 represents the time dependency of traffic in, for example, terminal device A, and curve k2 represents the time dependency of traffic in, for example, terminal device B different from terminal device A.

Referring to (a) of FIG. 7, in terminal device A, traffic is heavier at night than during the day (see curve k1). That is, in terminal device A, communication is less busy during the day and congested at night. Referring to (b) of FIG. 7, in terminal device B, traffic is heavier during the day than at night (see curve k2). That is, in terminal device B, communication is congested during the day and less busy at night.

Therefore, the scan scheduler 18 detects a time period during which communication is not busy for the M terminal devices that are the targets of the network scan, and determines the scan timing t _S (or the scan timing t' _S ) based on the detected time period.

The network scanner 11 receives the scan schedule SCH_scan from the scan scheduler 18 and performs a network scan on M terminal devices in accordance with the received scan schedule SCH_scan.

In this case, network scanning is performed without monitoring background traffic, or with monitoring background traffic.

[Update known scan response delay {RTT _m' _known}_z]
The update of the known scan response delay {RTT _m' _known}_z will be described. Here, z is an argument indicating the known estimated result of the network scan, and is an integer satisfying 1≦z≦Z. Furthermore, Z is the total number of known estimated results of the network scan. In this embodiment of the invention, the known estimated results of the network scan are "normal", "poor communication quality", and "network congestion", so z=1, 2, 3.

The update means 14 reads out scan response delays RTT ₁₁ to RTT _M3 from the correspondence table TBL1 or correspondence table TBL2 stored in the host information database 13, and reads out scan response delays {RTT _m' _known}_1 (m' = 1' to M') to {RTT m _' _known}_3 (m' = 1' to M') from the correspondence table TBL3 stored in the host information database 13.

The first numbers "1" to "M" in the subscripts "11" to "M3" of the scan response delays RTT ₁₁ to RTT _M3 are arguments for the terminal devices, and the second numbers "1" to "3" in the subscripts "11" to "M3" are arguments for the port numbers in each terminal device.

On the other hand, as described above, there are actually multiple scan response delays {RTT _m' _known}_1 (m' = 1' to M') corresponding to multiple ports in one IP address Add _m' . The same is true for scan response delays {RTT _m' _known}_2 (m' = 1' to M') and scan response delays {RTT _m' _known}_3 (m' = 1' to M').

Therefore, in describing the update of the known scan response delay {RTT _m' _known}_z, the scan response delays RTT ₁₁ to RTT _M3 read out from the correspondence table TBL1 or the correspondence table TBL2 are represented by the scan response delay {RTT _m,PNr _known} (m=1 to M), the scan response delay {RTT _m' _known}_1 (m'=1' to M') is represented by the scan response delay {RTT _m',PNr _known}_1 (m'=1' to M'), the scan response delay {RTT _m' _known}_2 (m'=1' to M') is represented by the scan response delay {RTT _m',PNr _known}_2 (m'=1' to M'), and the scan response delay {RTT _m' _known}_3 (m'=1' to M') is represented by the scan response delay {RTT _m',PNr _known}_3 (m'=1' to M').

The update means 14 reads out the scan response delay {RTT _{m, PNr} — unknown} (m=1 to M), the scan response delay {RTT _{m', PNr} — known} _ 1 (m'=1' to M'), the scan response delay {RTT _{m', PNr} — known} _ 2 (m'=1' to M') and the scan response delay {RTT _{m', PNr} — known} _ 3 (m'=1' to M'), and updates each of the scan response delays {RTT _{m, PNr —} unknown}, {RTT _{m', PNr} — known} _ 1, {RTT _{m', PNr} — known} _ 2, {RTT _{m', PNr} _known}_3 is normalized to be greater than or equal to "0" for all of the scan response delays {RTT _{m, PNr} _known} (m = 1 to M), {RTT _{m', PNr} _known}_1 (m' = 1' to M'), {RTT _{m', PNr} _known}_2 (m' = 1' to M'), and {RTT _{m', PNr} _known}_3 (m' = 1' to M').

Fig. 8 is a diagram showing an example of a normalized scan response delay RTT. Referring to Fig. 8, m/m' represents terminal device m or terminal device m'. Each of m and m' is, for example, 1 to 8. Also, PN ₁ to PN ₈ represent port numbers of each terminal device m (or each terminal device m').

Each row in Figure 8 represents eight normalized scan response delays (RTTs) when a network scan is performed on eight ports of one terminal device m (or one terminal device m').

In reality, a network scan was performed on 1,000 ports of each terminal device m (or terminal device m') to obtain 1,000 scan response delay RTTs. However, due to space limitations, in Figure 8, eight normalized scan response delays out of the 1,000 normalized scan response delay RTTs are shown for each terminal device m (or terminal device m').

In addition, in reality, network scans were performed on more than eight terminal devices m (or eight terminal devices m') to obtain scan response delay RTTs, but due to space limitations, Figure 8 shows normalized scan response delay RTTs for eight terminal devices m (or eight terminal devices m').

S ₁ to S ₄ are data sets of normalized scan response delay RTTs when the results of the network scan are different. In the embodiment of the present invention, as described above, the known estimated results of the network scan are three, namely, "normal", "poor communication quality", and "network congestion", so three of the data sets S ₁ to S ₄ are data sets of normalized scan response delay RTTs when the three known estimated results ("normal", "poor communication quality", and "network congestion") are obtained, and the remaining data set is a data set of normalized scan response delay RTTs when a network scan is performed in a communication environment where the results of the network scan are unknown.

The update means 14 normalizes each of the scan response delays {RTT _m, PNr_unknown}, {RTT _{m', PNr_known} }_1, {RTT _{m', PNr_known} }_2, and {RTT _{m', PNr_known} }_3 to be equal to or greater than "0" for all of the scan response delays {RTT _{m, PNr_unknown} } (m=1 to M), {RTT _{m', PNr_known} }_1 (m'=1' to M'), {RTT _{m', PNr_known} }_2 (m'=1' to M'), and {RTT _{m', PNr_known} }_3 (m'=1' to M'), to obtain the data sets S ₁ to S Create ₄ .

After creating the data sets S ₁ to S ₄ , the update means 14 selects one data set (for example, data set S ₁ ) from the data sets S ₁ to S ₄ .

Then, the update means 14 sequentially executes the following steps (i) to (iii) for the data set _S1 .
(i) For each of the eight scan response delays RTT in one row of the data set _S1 , check whether the scan response delay RTT is less than "0".
(ii) Delete any scan response delay RTT that is less than "0".
(iii) Execute (i) and (ii) for all eight rows (i.e., eight terminal devices).

The update means 14 sequentially executes the above steps (i) to (iii) for each of the data sets S ₂ to S ₄ .

Thereafter, the update means 14 outputs the data sets S ₁ to S ₄ to the calculation means 16. The calculation means 16 receives the data sets S ₁ to S ₄ from the update means 14 and calculates a baseline scan response delay b _RTT based on the scan response delays RTT included in the received data sets S ₁ to S ₄ .

The baseline scan response delay b _RTT is a scan response delay RTT whose distribution has been changed based on an "upper limit" and a "lower limit" so as to more easily produce a similarity or difference in distribution between a scan response delay RTT_unknown obtained when a network scan is performed in a communication environment where the results of the network scan are unknown, and a scan response delay RTT_known obtained when a network scan is performed in a communication environment where the results of the network scan are known.

A method for calculating the baseline scan response delay _bRTT will be described. The calculation means 16 selects a data set _S1 , and extracts eight scan response delays RTT in the first row (m/m'=1) from the selected data set _S1 . Then, the calculation means 16 calculates the average μ and standard deviation σ of the eight scan response delays RTT.

In this case, calculation means 16 subtracts the average from one scan response delay RTT, calculates the square (square) of the subtraction result (RTT-μ) for all eight scan response delays RTT, calculates the sum of the eight squares (RTT-μ) ² , and divides the sum of the eight squares (RTT-μ) ² by the number n (=8) of scan response delays RTT to calculate the square root [(sum of eight squares (RTT-μ) ² )/n] ^1/2 of the division result [(sum of eight squares (RTT-μ) ² )/n], thereby calculating the standard deviation σ.

Thereafter, the calculation means 16 calculates a score z _scr , which is a standard score of the scan response delay RTT, by the following formula.

In formula (1), x is one scan response delay RTT. μ/σ in formula (1) is the inverse of the coefficient of variation CV of the distribution of multiple scan response delay RTTs acquired by a network scan for one terminal device m (one terminal device m′). Therefore, the calculation means 16 calculates the score z _scr by subtracting the inverse of the coefficient of variation CV from one scan response delay RTT.

There are two methods for calculating the score z _scr using equation (1):
(Method 1)
One scan response delay RTT is randomly selected from the eight scan response delays RTT in the first row (m/m' = 1), and the selected one scan response delay RTT is substituted for x in equation (1) to calculate one score z _scr .
(Method 2)
Each of the eight scan response delays RTT in the first row (m/m′=1) is substituted for x in equation (1) to calculate eight scores z _scr .

After calculating the score z _scr , the calculation means 16 substitutes the score z _scr , the mean μ, the standard deviation σ, and the number of samples n (the number of scan response delays RTT in the first row (m/m' = 1) of the data set _S1 ) into equation (2) to calculate a lower limit L _b , and substitutes the score z _scr , the mean μ, the standard deviation σ, and the number of samples n into equation (3) to calculate an upper limit U _b .

When the calculation means 16 calculates the score z _scr using the above method 1, it calculates a lower limit L _b and an upper limit U _b for one randomly selected scan response delay RTT using equations (2) and (3), respectively, and calculates one baseline scan response delay b _RTT consisting of the calculated set {L _b , U _b } for one terminal device m (or one terminal device m').

On the other _hand , when the calculation means 16 calculates eight scores z _scr by the above method 2, it substitutes one of the eight scores z _scr , the mean μ, the standard deviation σ and the number of samples n into equations (2) and (3) to calculate a lower limit L _b and an upper limit U _b , respectively, and calculates one baseline scan response delay b _RTT consisting of the calculated set {L _b , U _b } for all of the 1st to 8th scan response delays RTT in the first row (m/m' = 1), thereby calculating eight baseline scan response delays b _RTT1 to b _RTT8 (= {L _b1 , U _b1 } to L _b8 , U _b8 }) for one terminal device m (or one terminal device m').

The calculation means 16 calculates one baseline scan response delay b RTT or eight baseline scan response delays b RTT1 to b _RTT8 (= {L b1 , U b1 } to L b8 , U b8 }) for _one terminal device m (or one terminal device m') using the above-mentioned method based on the eight scan response delays _RTT in each of the second row (m _{/ m} ' = 2) to the eighth _row (m _/ m _' = 8) of the dataset S ₁ , for all rows from the second row (m/m' = 2) to the eighth row (m/m' = 8) of the dataset S 1.

Then, for each of the data sets _S2 to _S4 , the calculation means 16 calculates one baseline scan response delay _bRTT or eight baseline scan response delays _bRTT1 to _bRTT8 (= { _{Lb_1} , _{Ub_1} } to _{Lb_8} , _{Ub_8} }) for one terminal device m (or one terminal device m') using the method described above.

FIG. 9 is a diagram showing the score z _scr and the baseline scan response delay b _RTT calculated for one terminal device.

FIG. 9 shows the score z _{scr and} the baseline scan response delay b RTT when the score z scr is calculated by the above method 1 or method 2, using as an example the eight scan response delays RTT for the terminal device m/m _′ =1 in the first row of the data set S 1 shown in _FIG . 8 .

Referring to FIG. 9, when the calculation means 16 calculates the score z _scr by the above method 1, the calculation means 16 randomly selects, for example, a scan response delay RTT of “0.012949” corresponding to the port number PN 1 from among the eight scan response delay RTTs corresponding to _the port numbers PN ₁ to PN ₈ , respectively.

Then, the calculation means 16 substitutes the scan response delay RTT of "0.012949" into formula (1) to calculate the score z _scr . After that, the calculation means 16 substitutes the score z _scr into formulas (2) and (3) to calculate the lower limit L _b and the upper limit U _b , respectively. As a result, one baseline scan response delay b _RTT = {L _b , U _b } is calculated corresponding to one score z _scr .

On the other hand, when calculating score z _scr by the above method 2, calculation means 16 sequentially selects eight scan response delays RTT corresponding to port numbers PN ₁ to PN ₈ , respectively, and calculates eight scores z _scr1 to z _scr8 by formula (1). Then, calculation means 16 substitutes the eight scores z _scr1 to z _scr8 into formulas (2) and (3) to sequentially calculate lower limit L _b and upper limit U _b , respectively. As a result, eight baseline scan response delays b _RTT1 to b _RTT8 are calculated corresponding to scores z _scr1 to z _scr8 , respectively.

In this case, a score z _scr1 is calculated for a scan response delay RTT of "0.012949", and a baseline scan response delay b _RTT1 = {L _b1 , U _b1 } is calculated corresponding to the score z _scr1 . A score z _scr2 is calculated for a scan response delay RTT of "0.009000", and a baseline scan response delay b _RTT2 = {L _b2 , U _b2 } is calculated corresponding to the score z _scr2 . Similarly, a score z _scr8 is calculated for a scan response delay RTT of "0.013000", and a baseline scan response delay b _RTT8 = {L _b8 , U _b8 } is calculated corresponding to the score z _scr8 .

FIG. 10 is a diagram illustrating an example of a baseline scan response delay _bRTT calculated by randomly selecting one scan response delay RTT.

Referring to FIG. 10, baseline scan response delays _bRTT1 through _bRTT4 are associated with data sets _S1 through _S4 , respectively.

In the baseline scan response delay b _RTT1 , the lower limit L _b = 0.011943 and upper limit U _b = 0.009294 corresponding to m/m' = 1 are the lower limit L b and upper limit U b calculated using equations (1) to (3) by randomly selecting one scan response delay RTT from _the eight scan response delays RTTs associated with m/ _m' = 1 in the data set S ₁ of Figure 8.

In addition, for the baseline scan response delay b _RTT1 , the lower limit L _b =0.011661 and upper limit U _b =0.009076 corresponding to m/m' = 2 are the lower limit L b and upper limit U b calculated using equations ( ₁ ) to (3) by randomly selecting one scan response delay RTT from the eight scan response delays RTTs associated with m/ _m ' = 2 in the data set _S1 of Figure 8.

Similarly, in the baseline scan response delay b _RTT1 , the lower limit L _b =0.012224 and the upper limit U _b =0.009513 corresponding to m/m'=3 are calculated by randomly selecting one scan response delay RTT from the eight scan response delays RTTs associated with m/m'=3 in the data set S ₁ of FIG. 8 and using equations (1) to (3). In the baseline scan response delay b _RTT1 , the lower limit L _b =0.012922 and the upper limit U _b =0.010065 corresponding to m/m'=4 are calculated by randomly selecting one scan response delay RTT from the eight scan response delays RTTs associated with m/m'=4 in the data set S ₁ of FIG. 8 and using equations (1) to (3). In the baseline scan response delay b _RTT1 , the lower limit L _b =0.019816 and the upper limit U b =0.010 In the baseline scan response delay b RTT1, the _lower limit _{L b} =0.019956 and the upper limit U b =0.015532 corresponding to m/m'=6 are calculated by randomly selecting one scan response delay RTT from the eight scan response delays RTTs corresponding to m/m'=6 in the data set S ₁ of FIG. 8 and using equations (1) to (3). In the baseline scan response delay b _RTT1 , the lower limit L _b =0.020095 and the upper limit U _b =0.015642 corresponding to m/m'=7 are calculated by randomly selecting one scan response delay _RTT from the eight scan response delays _RTTs corresponding to m/m'=6 in the data set _S 1 of FIG. In the baseline scan response delay b RTT1, a lower limit L b =0.019395 and an upper limit _{U b} =0.015092 corresponding to _m /m'=8 in the data set _S 1 of FIG. 8 are calculated using equations (1) to (3) by randomly selecting one scan response delay RTT from the eight scan response delays RTTs corresponding to m/m'=8 in the data set S ₁ of FIG. 8 .

Similarly, the lower limit L _b and upper limit U _b of the baseline scan response delay b _RTT2 corresponding to the data set _S2 , the lower limit L _b and upper limit U _b of the baseline scan response delay b _RTT3 corresponding to the data _{set S3} , and the lower limit L _b and upper limit U _b of the baseline scan response delay b _RTT4 corresponding to the data set S4 are each calculated using equations ( ₁ ) to ( ₃ ) by randomly selecting one scan response delay RTT from the eight scan response delays RTTs for m/m' = 1 to 8 in each of the data sets S 2 to S 4 shown in FIG.

As shown in FIG. 10, when one scan response delay RTT is randomly selected to calculate a baseline scan response delay b _RTT , one baseline scan response delay b _RTT is calculated for one terminal device.

Note that the value of the upper limit U _b is smaller than the value of the lower limit L _b for each of the baseline scan response delays b _RTT1 to b _RTT4 shown in Figure 10. This is because the score z _scr is a negative value.

11 is a diagram showing an example of a baseline scan response delay b _RTT calculated using all of the multiple scan response delays RTT corresponding to the multiple ports of one terminal device. Note that in FIG. 11, the number of ports in one terminal device is eight.

11, baseline scan response delays _bRTT1 to _bRTT8 are associated with port numbers _PN1 to _PN8 , respectively.

In the data set _S1 , the lower limit L _b =0.011943 and the upper limit U b =0.009294 of the baseline scan response delay b _RTT1 corresponding to m/m'=1 and PN ₁ are the lower limit L _b and upper limit U _b calculated by equations (1) to (3) by selecting the scan response delay RTT of "0.012949" corresponding to m/m'= ₁ and PN ₁ in the data set _S1 of Figure 8.

In addition, in the data set _S1 , the lower limit L _b =0.011944 and the upper limit U _b =0.009293 of the baseline scan response delay b _RTT2 corresponding to m/m' =1 and PN ₂ are the lower limit L _b and upper limit U _b calculated by equations ( ₁ ) to (3) by selecting the scan response delay RTT of "0.009000" corresponding to m/m' =1 and PN ₂ in the data set S1 of Figure 8.

Furthermore, in the data set _S1 , the lower limit L _{b =0.011944 and the upper limit U b} =0.009293 of the baseline scan response delay b _RTT3 corresponding to m/m' =1 and PN ₃ are the lower limit L b and upper limit U _b calculated by equations ( ₁ ) to (3) by selecting the scan response delay RTT of "0.010000" corresponding to m/ _m ' =1 and PN ₃ in the data set _S1 of FIG. 8.

Furthermore, in the data set _S1 , the lower limit L _b =0.011944 and the upper limit U _b =0.009293 of the baseline scan response delay b _RTT4 corresponding to m/m' =1 and PN ₄ are the lower limit L b and upper limit U _b calculated by equations ( ₁ ) to (3) by selecting the scan response delay RTT of "0.010000" corresponding to m/m' =1 and PN ₄ in the data set _S1 of FIG. 8.

Furthermore, in the data set _S1 , the lower limit L _{b =0.011944 and the upper limit U b} =0.009293 of the baseline scan response delay b _RTT5 corresponding to m/m' = 1 and PN ₅ are the lower limit L b and upper limit U _b calculated by equations ( ₁ ) to (3) by selecting the scan response delay RTT of "0.010000" corresponding to m/ _m ' = 1 and PN ₅ in the data set _S1 of FIG. 8.

Furthermore, in the data set _S1 , the lower limit L b =0.011943 and the upper limit U _b =0.009294 of the baseline scan response delay b _RTT6 corresponding to m/m' =1 and PN ₆ are the lower limit L _b and upper limit U _b calculated by equations ( ₁ ) to (3) by selecting the scan response delay RTT of "0.012000" corresponding to m/m' =1 and PN ₆ in the data set _S1 of Figure 8.

Furthermore, in the data set _S1 , the lower limit L _{b =0.011944 and the upper limit U b =} 0.009293 of the baseline scan response delay b _RTT7 corresponding to m/m'=1 and PN ₇ are the lower limit L b and upper limit U _b calculated by equations ( ₁ ) to (3) by selecting the scan response delay RTT of "0.008000" corresponding to m/m'=1 and PN ₇ in the data set _S1 of FIG. 8.

Furthermore, in the data set _S1 , the lower limit L b =0.011943 and the upper limit U _b =0.009294 of the baseline scan response delay b _RTT8 corresponding to m/m' = 1 and PN ₈ are the lower limit L _b and upper limit U _b calculated by equations ( ₁ ) to (3) by selecting the scan response delay RTT of "0.013000" corresponding to m/m' = 1 and PN ₈ in the data set _S1 of Figure 8.

In this way, the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m'=1 and _PN1 to _PN8 , respectively, of data set _S1 are each composed of a lower limit Lb and an upper limit _Ub calculated by equations ( ₁ ) to (3) by selecting eight scan response delays RTT corresponding to m/m'=1 and _PN1 to _PN8 , respectively, of data set _S1 in FIG.

Similarly, the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m'=2 and _PN1 to _PN8 , respectively, of data set _S1 are composed of lower limit Lb and upper limit _Ub calculated by equations ( ₁ ) to (3) by selecting eight scan response delays RTT corresponding to m/m'=2 and _PN1 to PN8, respectively, of data set _S1 in Figure ₈ .

In addition, the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m'=3 and _PN1 to _PN8 in the data set _S1 , respectively, are composed of a lower limit Lb and an upper limit _Ub calculated by equations ( ₁ ) to (3) by selecting eight scan response delays RTT corresponding to m/m'= ₃ and _PN1 to PN8 in the data set S1 in Figure ₈ , respectively.

Furthermore, the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m'=4 and _PN1 to _PN8 , respectively, of the data set _S1 are composed of a lower limit Lb and an upper limit _Ub calculated by equations ( ₁ ) to (3) by selecting eight scan response delays RTT corresponding to m/m'=4 and _PN1 to _PN8 , respectively, of the data set _S1 in FIG.

Furthermore, the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m'=5 and _PN1 to _PN8 , respectively, of the data set _S1 are composed of a lower limit Lb and an upper limit _Ub calculated by equations ( ₁ ) to (3) by selecting eight scan response delays RTT corresponding to m/m'=5 and _PN1 to _PN8 , respectively, of the data set _S1 in FIG.

Furthermore, the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m'=6 and _PN1 to _PN8 , respectively, of the data set _S1 are composed of a lower limit Lb and an upper limit _Ub calculated by equations ( ₁ ) to (3) by selecting eight scan response delays RTT corresponding to m/m'=6 and _PN1 to _PN8 , respectively, of the data set _S1 in FIG.

Furthermore, the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m' = 7 and PN ₁ to PN ₈ , respectively, of the data set _S1 are composed of a lower limit Lb and an upper limit Ub calculated by equations ( ₁ ) to (3) by selecting eight scan response delays RTT corresponding to m/m' = ₇ and PN ₁ to PN ₈ , respectively, of the data set _S1 in Figure 8.

Furthermore, the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m'=8 and _PN1 to _PN8 , respectively, of the data set _S1 are composed of a lower limit Lb and an upper limit _Ub calculated by equations ( ₁ ) to (3) by selecting eight scan response delays RTT corresponding to m/m'=8 and PN1 _{to PN8} , respectively, of the data set _S1 in FIG.

The same is true for baseline scan response delays _bRTT1 through _bRTT8 in data sets _S2 through _S4 .

As shown in FIG. 11, when the baseline scan response delay bRTT is calculated using all eight scan response delays RTT corresponding to the eight ports in one terminal device _, eight baseline scan response delays _bRTT1 to _bRTT8 are calculated for one terminal device m (or one terminal device m').

Note that for each of the baseline scan response delays b _RTT1 to b _RTT8 shown in FIG. 11, the scores z _scr1 to z _scr8 are negative values, and therefore the value of the upper limit U _b is smaller than the value of the lower limit L _b .

In an embodiment of the present invention, the baseline scan response delay _bRTT may be calculated using the score _zβ instead of the score _zscr .

In this case, the calculation means 16 first selects a confidence level. The confidence level is set to, for example, 95%.

When the calculation means 16 selects the confidence level, it calculates β by β = 1 - (confidence level/100). When the confidence level is 95%, the calculation means 16 calculates β by β = 1 - (95/100) = 0.05.

Thereafter, the calculation means 16 calculates the critical probability p ^* by p ^* = 1 - (β/2). When β is 0.05, the calculation means 16 calculates the critical probability p ^* by p ^* = 1 - (0.05/2) = 0.975.

12 is a diagram showing a z table. When the calculation means 16 calculates the critical probability p ^* , it refers to the z table, detects the z value (=1.9) in the column direction corresponding to the critical probability p ^* (=0.975) and the z value (=0.06) in the row direction corresponding to the critical probability p ^* (=0.975), and calculates the score _zβ of "1.96" by adding the two detected z values ("1.9" and "0.06"). Note that the calculation means 16 holds the z table in advance when calculating the score _zβ .

The z table indicates the probability that the z value calculated by (RTT-μ)/σ is equal to or greater than a certain value when the scan response delay RTT follows a standard normal distribution. In the above example, the probability that the z value is equal to or greater than "1.96" is 97.5%. In this embodiment of the invention, the critical probability p ^* (=0.975) is calculated based on the confidence level (=95%) to be achieved, and the z value (=1.96) when the calculated critical probability p ^* (=0.975) is obtained is calculated based on the z table. The z value (=1.96) when the critical probability p ^* (=0.975) is obtained is set as the score z _β . The critical probability p ^* of "0.975" is obtained for a z value of 1.96 or greater, so the z value of 1.96 or greater is the confidence interval. Therefore, the score z _β consists of the confidence interval when the confidence level (=95%) is achieved.

After calculating the score z _β , the calculation means 16 calculates the standard error SE by the following formula.

Next, the calculation means 16 calculates the degree of freedom df using the following formula:

In equations (4) and (5), n is the number of scan response delays RTT in one terminal device m (or one terminal device m').

After calculating the standard error SE and the degrees of freedom df, the calculation means 16 substitutes the score z _β , the standard error SE, and the degrees of freedom df into equations (6) and (7) to calculate the lower limit L _b and the upper limit U _b, respectively.

When calculating the lower limit L _b and the upper limit U _b using the score z _β , multiple scan response delays RTT in one terminal device m (or one terminal device m') are used only to calculate the mean μ and standard deviation σ, so that one baseline scan response delay b _RTT (consisting of the lower limit L _b and the upper limit U _b ) is calculated for one terminal device m (or one terminal device m').

In the embodiment of the present invention, z _scr ×(σ/n) in formulas (2) and (3) constitutes a "predetermined value", and z _β ×(SE/(df) ^1/2 ) shown in formulas (6) and (7) constitutes a "predetermined value". Then, as shown in formula (1), z _scr is the result of subtracting the reciprocal (μ/σ) of the coefficient of variation (σ/μ) in the distribution of multiple scan response delays x (=RTT) from the scan response delay x (=RTT). Furthermore, the coefficient of variation (σ/μ) represents the relative variation of multiple scan response delays x (=RTT). Therefore, z _scr ×(σ/n) is calculated based on the relative variation (i.e., the coefficient of variation (σ/μ)) of multiple scan response delays x (=RTT). As a result, the "predetermined value" is calculated based on the relative variation (i.e., the coefficient of variation (σ/μ)) of multiple scan response delays x (=RTT).

FIG. 13 shows the baseline scan response latency _bRTT calculated using the score _zβ .

With reference to FIG. 13, baseline scan response delays b _RTT1 through b _RTT4 are associated with data sets S ₁ through S ₄ , respectively.

For the baseline scan response delay b _RTT1 , the lower limit L _b =0.010168 and upper limit U _b =0.011069 corresponding to m/m'=1 are the lower limit L b and upper limit U b calculated using equations (4) to (7) based on the number n, mean μ, and standard deviation σ of the _eight scan response delays RTTs associated with m/m'= ₁ in the data set S ₁ of Figure 8, and the score z _β .

In addition, for the baseline scan response delay b _RTT1 , the lower limit L _b =0.009832 and upper limit U _b =0.010905 corresponding to m/m' = 2 are the lower limit L b and upper limit U b calculated using equations ( ₄ ) to (7) based on the number n, mean μ, and standard deviation σ of the eight scan response delays RTTs associated with m/m _' = 2 in the data set S 1 of Figure 8, and _the score z _β .

Similarly, in the baseline scan response delay b _RTT1 , the lower limit L _b =0.010339 and the upper limit U _b =0.011398 corresponding to m/m'=3 are calculated by equations (4) to (7) based on the number n, average μ, and standard deviation σ of the eight scan response delays RTTs associated with m/m'=3 in the data set S ₁ of FIG. 8, and the score z _β . In the baseline scan response delay b _RTT1 , the lower limit L _b =0.010609 and the upper limit U _b =0.012378 corresponding to m/m'=4 are calculated by equations (4) to (7) based on the number n, average μ, and standard deviation σ of the eight scan response delays RTTs associated with m/m'=4 in the data set S ₁ of FIG. 8, and the score z _β . In the baseline scan response delay b RTT1, the lower limit L b =0.010609 and the upper limit U b =0.012378 corresponding to m/m'=5 are calculated by equations (4) to (7) based on the number n, average μ, and standard deviation σ of the eight scan response delays _RTTs associated with m/m'=4 in the data set S 1 of _FIG. 8, the lower limit L b =0.017133 and the upper limit U _b =0.018105 are calculated by equations ( ₄ ) to (7) based on the number n, average μ, and standard deviation σ of the eight scan response delays RTTs associated with m/m'=5 in the data set S 1 of FIG. 8, and the score z _β . In the baseline scan response delay b _RTT1 , the lower limit L _b =0.017165 and the upper limit U _b =0.018322 corresponding to m/m'=6 are calculated by equations (4) to (7) based on the number n, average μ, and standard deviation σ of _{the eight} scan response delays RTTs associated with m/m'=6 in the data set S 1 of FIG. 8, and the score z _β . In the baseline scan response delay b _RTT1 , the lower limit L _b =0.017225 and the upper limit U _b 8. In the baseline scan response delay b RTT1, the lower limit L b =0.016817 and the upper limit U b =0.017670 corresponding to m/m'= ₈ are calculated by equations (4) to (7) based on the number _n , average μ, and standard deviation _σ of the eight scan response delays _RTT associated with m/m'=8 in the dataset S ₁ of FIG. 8, and _{the score z β .}

Similarly, the lower limit L _b and upper limit U _b of the baseline scan response delay b _RTT2 corresponding to the data set _S2 , the lower limit L _b and upper limit U _b of the baseline scan response delay b _RTT3 corresponding to the data set _S3 , and the lower limit L _b and upper limit U _b of _the baseline scan response delay b _RTT4 corresponding to the data set S4 are calculated using equations ( ₄ ) to ( ₇ ) based on the number n, average μ, and standard deviation σ of the eight scan response delays RTTs for m/m' = 1 to 8 in each of the data sets S2 to S4 shown in FIG. 8, and the score z _β .

As shown in FIG. 13, when calculating the baseline scan response delay b _RTT using the score z _β , the same score z _β is used for each of the data sets S ₁ to S ₄ , and one baseline scan response delay b _RTT is calculated for one terminal device in each of the data sets S ₁ to S ₄ .

Note that the baseline scan response delays b _RTT1 to b _RTT4 shown in Figure 10, the baseline scan response delays b _RTT1 to b _RTT8 shown in Figure 11, and the baseline scan response delays b _RTT1 to b _RTT4 shown in Figure 13 are actually calculated using the number n, average μ, and standard deviation σ of 1,000 scan response delays RTT corresponding to 1,000 ports in one terminal device m (or one terminal device m').

Table 1 shows the calculation method of the baseline scan response delay b _RTT described above and the number of baseline scan response delays b _RTT per terminal device m (per terminal device m').

As shown in Table 1, the score z _scr is composed of one z _{scr_c} , R (R=PN _{r — MAX} ) scores z _{scr — 1} to z _{scr_R} , and one z _β .

When the score z _scr consists of one z _{scr_c} , the score z _{scr_c} is calculated using one scan response delay RTT (=x _c ) randomly selected from R scan response delays RTTs corresponding to R ports in one terminal device. Then, each of the lower limit L _b and the upper limit U _b is calculated using one score z _{scr_c} . Therefore, the baseline scan response delay b _RTT per one terminal device is one ({L _b , U _b }).

When the score z _scr consists of R scores z _{scr — 1} to z _{scr_R} , z _{scr —} 1 to z _{scr_R} are calculated using R scan response delays RTTs (= x ₁ to x _R ) corresponding to R ports in one terminal device. Then, each of the lower limit L _{b_r} and upper limit U _{b_r} is calculated using the R scores z _{scr — 1} to z _{scr_R} . Therefore, the baseline scan response delays b _RTTs per terminal device are R ({L _b1 , U _b1 }, L _b2 , U _b2 }, ..., L _bR , U _bR }).

When the score z _scr is composed of one z _β , the score z _β is calculated based on the critical probability p ^* calculated based on the confidence level and the z table. Then, each of the lower limit L _b and the upper limit U _b is calculated using one score z _β . Therefore, the baseline scan response delay b _RTT per terminal device is one ({L _b , U _b }).

The calculation means 16 calculates the baseline scan response delay bRTT for all of the scan response delays {RTT _m, PNr_known} (m=1 to M), scan response delays {RTT _{m', PNr_known} }_1 (m'=1' to M'), scan response delays {RTT _{m', PNr_known} }_2 (m'=1' to M') and scan response delays {RTT _{m', PNr_known} }_3 (m'=1' to M' ₎ using the method described above.

Here, the baseline scan response delay b _RTT of the scan response delay {RTT _m,PNr _unknown} (m=1 to M) is denoted as the baseline scan response delay {b _RTT _ _m,PNr _unknown} (m=1 to M), the baseline scan response delay b _RTT of the scan response delay {RTT _m',PNr _known}_1 (m'=1' to M') is denoted as the baseline scan response delay {b _RTT _ _m',PNr _known}_1 (m'=1' to M'), and the baseline scan response delay b _RTT of the scan response delay {RTT _m',PNr _known}_2 (m'=1' to M') is denoted as the baseline scan response delay {b _RTT _ _m',PNr _known}_2 (m'=1' to M'), and the baseline scan response delay b _RTT of the scan response delay {RTT _m',PNr _known}_3 (m'=1' to M') is denoted as baseline scan response delay {b _RTT _ _m',PNr _known}_3 (m'=1' to M').

Then, the calculation means 16 calculates a mean absolute deviation W _unknown based on the baseline scan response delay {b _RTT _ _{m, PNr} _ known} (m = 1 to M), calculates a mean absolute deviation W known _ 1 based on the baseline scan response delay {b _RTT _ _{m', PNr} _ known} _ 1 (m' = 1' to M'), calculates a mean absolute deviation W _{known _ 2} based on the baseline scan response delay {b _RTT _ _{m', PNr} _ known} _ 2 (m' = 1' to M'), and calculates a mean absolute deviation W _{known _} 3 based on the baseline scan response delay {b _RTT _ _{m', PNr} _ known} _ 3 (m' = 1' to M').

The mean absolute deviations W _unknown , W _{known — 1} , W _{known — 2} , and W _{known — 3} are calculated by the following formula.

In equation (8), W represents the mean absolute deviation, M _a represents the number of terminal devices m/m' consisting of M or M', x _{i _ bRTT} represents the baseline scan response delay b _RTT , and m(Mo) represents the value of the baseline scan response delay b _RTT that is most likely to occur.

A method for calculating m(Mo) will be described. Fig. 14 is a diagram showing an example of a baseline scan response delay b _RTT . Fig. 15 is a diagram showing a result of counting the number of samples of each value of the baseline scan response delay b _RTT shown in Fig. 14. Fig. 16 is a diagram showing the distribution of the baseline scan response delay b _RTT shown in Fig. 15.

In order to make it easier to understand how to calculate m(Mo), Fig. 14 shows the results of rounding the baseline scan response delays _bRTT1 to _bRTT4 shown in Fig. 10 to four decimal places. As shown in Fig. 14, the baseline scan response delays _bRTT1 to _bRTT4 have values in the range of [0, 0.021].

As a result of counting the number of samples of each value of the baseline scan response delays b _RTT1 to b _RTT4 shown in FIG. 14, the baseline scan response delay b RTT of 0.009 is "7", the baseline scan response delay b _RTT of 0.010 is "6", the baseline scan response delay b RTT of 0.012 is "9", the baseline scan response delay b _RTT of 0.013 is "7", the baseline scan response delay b RTT of 0.015 is "12", the baseline scan response delay b RTT of 0.016 is "5", the baseline scan response delay b RTT of 0.017 is "2", the baseline scan response delay b _RTT of 0.019 is "4", the baseline scan response delay b _RTT of 0.020 is "10", the baseline scan response delay b RTT of 0.021 is "13", the baseline scan response delay b RTT of 0.022 is "14", the baseline scan response delay b RTT of 0.026 is "15", the baseline scan response delay b _RTT of 0.028 is "16", the baseline scan response delay b _RTT of 0.029 is "17", the baseline scan response delay b RTT of 0.030 is "18", the baseline scan response delay b _RTT of 0.032 is "19", the baseline scan response delay b RTT of 0.036 is "20", the baseline scan response delay b RTT of 0.034 is "21", the baseline scan response delay b RTT of 0.036 is "22", the baseline scan response delay b _RTT of 0.036 is "23", the baseline scan response delay b RTT of 0.036 is "24", the baseline scan response delay b _{RTT of 0.032 is "25", the baseline scan response delay b RTT of 0.036 is "36", the baseline scan response delay b RTT} of 0.036 is "38", the The other baseline scan response delay _bRTT is "0" (see FIG _. 15).

The number of samples for each value of baseline scan response delay _bRTT can be plotted as shown in FIG.

Fig. 17 is a diagram showing the results when some values of the baseline scan response delay b _RTT shown in Fig. 14 are changed. Fig. 18 is a diagram showing the distribution of the baseline scan response delay b _RTT shown in Fig. 17.

17, in order to make it easier to understand how to calculate m(Mo), the value corresponding to m/m'=8 in the upper limit _Ub of the baseline scan response delay _bRTT4 was changed from "0.015" to "0.020" (see circle). As a result, as shown in FIG 18, the number of samples for "0.015" becomes 11, and the number of samples for "0.020" becomes 11.

Fig. 19 is a diagram showing the distribution of baseline scan response delay _bRTT when the scale of the horizontal axis in Fig. 18 is changed to "0.005" units. Fig. 20 is a diagram showing the correspondence relationship between the range of baseline scan response delay _bRTT and the number of samples generated. Fig. 21 is a diagram showing a specific distribution of the probability density function of the baseline scan response delay.

If the number of samples of each specific value in the specific distribution of baseline scan response delay b _RTT shown in Fig. 18 is counted and the number of samples for each specific value is displayed, the result is as shown in Fig. 19. As a result, it can be seen from Fig. 19 that the baseline scan response delay b _RTT is most likely to occur at probability density values of 0.015 and 0.020.

As shown in Figure 20, the number of samples generated in the PDF region of 0.0 to 0.005, the PDF region of 0.0051 to 0.010, the PDF region of 0.011 to 0.015, the PDF region of 0.0151 to 0.020, the PDF region of 0.021 to 0.025, and the PDF region of 0.0251 to 0.030 are "0", "13", "27", "22", "2", and "0", respectively.

The correspondence between the PDF regions and the number of samples generated in each region is as shown in Figure 21. The number of samples (=22) generated in the PDF region of 0.0151 to 0.020 is r, the interval (=0.005) of 0.0151 to 0.020 is h, 0.015 is L, the number of samples (=27) generated in the PDF region of 0.011 to 0.015 is _r1 , and the number of samples (=2) generated in the PDF region of 0.0211 to 0.025 is _r2 . Therefore, the mode m (Mo) can be calculated by substituting r (=22), _r1 (=27), _r2 (=2), h (=0.005), and L (=0.015) into the following formula.

In formula (9), L indicates the lower boundary of the probability density function region (=PDF (Probability Density Function) region) including two values of the most likely baseline scan response delay b _RTT when the result of the network scan is unknown or when the result of the network scan is known. Also, r indicates the number of samples (=the number of baseline scan response delays b RTT) of the most likely PDF region when the result of the network scan is unknown or when the result of the network scan is known. Furthermore, r ₁ indicates the number of samples (=the number of baseline scan response delays b _RTT ) of the previous PDF region in the direction where the baseline scan response delay b _RTT is smaller than the most likely PDF region when the result of the network scan is unknown or when the result of the network scan is known. Furthermore, r ₂ indicates the number of samples (=the number of baseline scan response delays b _RTT ) of the next PDF region in the direction where the baseline scan response delay b _RTT is larger than the most likely PDF region when the result of the network scan is unknown or when the result of _the network scan is known. Furthermore, h denotes the interval between each PDF region (i.e., the width of each PDF region).

As shown in Fig. 21, r is the number of samples of the baseline scan response delay _bRTT in the PDF region REG _PDF surrounded by the two most likely values of the baseline scan response delay _bRTT (=0.015 and 0.020) and the curve showing the probability density function. Furthermore, L is the lower value (=0.015) of the two most likely values of the baseline scan response delay _bRTT (=0.015 and 0.020). Furthermore, h is the interval between the two values (=0.015 and 0.020). _r1 consists of the number of samples in a region of width h adjacent to the PDF region REG _PDF on the side where the baseline scan response delay b _RTT is smaller than that of the PDF region REG _PDF , and _r2 consists of the number of samples in a region of width h adjacent to the PDF region REG _PDF on the side where the baseline scan response delay b _RTT is larger than that of the PDF region REG _PDF .

Fig. 22 is a diagram showing another example of baseline scan response delay b _RTT . Fig. 23 is a diagram showing the distribution of baseline scan response delay b _RTT shown in Fig. 22. Fig. 24 is a diagram showing the distribution of baseline scan response delay b _RTT when the scale of the horizontal axis in Fig. 23 is changed to "0.005" units.

With reference to FIG. 22, in order to make it easier to understand the method of obtaining m(Mo), in the lower limit L _b of the baseline scan response delay b _RTT2 , the value corresponding to m/m'=1 is changed from "0.013" to "0.012" (see circle mark), in the lower limit L _b of the baseline scan response delay b _RTT4 , the value corresponding to m/m'=2 is changed from "0.013" to "0.012" (see circle mark), and in the upper limit U _b of the baseline scan response delay b _RTT4 , the value corresponding to m/m'=8 is changed from "0.015" to "0.020" (see circle mark). As a result, as shown in FIG. 23, there are three values of the baseline scan response delay b _RTT that are most likely to occur. In this case, the interval h is adjusted to the width of the PDF region including the three most likely values of the baseline scan response delay b _RTT . Then, the minimum and maximum values are detected from the three most likely values of the baseline scan response delay b _RTT . Then, using the minimum and maximum values as the two most likely baseline scan response delay _bRTT values, r, _r1 , _r2 , L and h are determined using the method described in Figures 18 to 21, and the determined r, _r1 , _r2 , L and h are substituted into equation (9) to calculate the mode m (Mo).

When there are four or more most likely baseline scan response delay _bRTT values, the mode m (Mo) is calculated in the same manner as when there are three most likely baseline scan response delay _bRTT values.

The calculation means 16 calculates the mode m(Mo)_known based on the baseline scan response delay {b _RTT — _{m, PNr} — known} (m=1 to M) by the method described with reference to Figures 18 to 21, calculates the mode m(Mo)_known_1 based on the baseline scan response delay {b _RTT — _{m', PNr —} known}_1 (m'=1' to M'), calculates the mode m(Mo)_known_2 based on the baseline scan response delay {b _RTT — _{m', PNr —} known}_2 (m'=1' to M'), and calculates the mode m(Mo)_known_3 based on the baseline scan response delay {b _RTT — _{m', PNr} — known}_3 (m'=1' to M').

Then, the calculation means 16 substitutes the baseline scan response delay {b _RTT _ _{m,PNr _} unknown} (m = 1 to M) and the mode m (Mo)_ unknown into equation (8B) to calculate C(x _{i _ bRTT} ), and then substitutes the calculated C(x _{i _} bRTT ) and the number of samples M of the baseline scan response delay {b _RTT _ _m,PNr _ unknown} (m = 1 to M) into equation (8A) to calculate the mean absolute deviation W _unknown .

In addition, the calculation means 16 calculates C(x i _bRTT ) by substituting the baseline scan response delay {b _RTT _m _{', PNr} _known}_1 (m'=1' to M') and the mode m(Mo)_known_1 into equation (8B), and calculates the mean absolute deviation W known_1 by substituting the calculated C(x _{i _bRTT} ) and the number of samples M' of the baseline scan response delay {b _RTT _m _{', PNr} _known} _{_1} (m'=1' to M') into equation (8A).

Furthermore, the calculation means 16 similarly calculates the mean absolute deviation W known_2 using equation (8) based on the baseline scan response delay {b _RTT _ m' _{, PNr} _known} _2 (m' = 1' to M') and the mode m(Mo)_known_2, and calculates the mean absolute deviation W _{known_3} using equation (8) based on the baseline scan response delay {b _RTT _ _{m', PNr} _known} _3 (m' = 1' to M') and the mode m(Mo) _{_known_3} .

Note that C(x _{i _ bRTT} ) calculated by equation (8B) is expressed by the following equation.

In equation (10), the equation on the first line means that C(a 1Lb ) and C( _{a 1Ub )} are calculated using the lower limit L _b (=0.012) and upper limit U _b (=0.009) corresponding to m/m′=1 for the baseline scan response delay b _RTT1 shown in FIG. 17 . The equation on the second line means that C(a _2Lb ) and C(a _2Ub ) are calculated using the lower limit L _b (=0.012) and upper limit U _b (=0.009) corresponding to m/m′=2 for the baseline scan response delay b RTT1 shown in FIG. 17 . Similarly, the equation on the last line means that C(a _ALb ) and C(a _1Ub ) are calculated using the lower limit L _b (=0.019) and upper limit U _b (=0.015) corresponding to m/m′=8 for the baseline scan response delay b _RTT1 shown in _FIG . 17 . ), C(a _AUb ). In other words, equation (10) means that in the baseline scan response delay b _RTT1 shown in FIG. 17, the baseline scan response delay {L _b , U _b } corresponding to m/m'=1, the baseline scan response delay {L _b , U b } corresponding to m/m'=2, ..., the baseline scan response delay {L _b , U _b } corresponding to m/m'= ₈ are sequentially substituted into equation (8B) to calculate C(x _{i _ bRTT} ). As a result, A indicates the total number of eight baseline scan response delays b RTT corresponding to m/m'=1 to 8 in each of the data sets S ₁ to S ₄ shown in _FIG . 17 (i.e., [the total number M (or M') of terminal devices in each of the data sets S ₁ to S ₄ ]).

As shown in FIG. 11 described above, when a plurality of baseline scan response delays b _RTT1 to b _RTT8 are calculated for one terminal device m (or one terminal device m'), in equation (10), the equation in the first line means that C(a 1Lb ) and C(a 1Ub ) are calculated using the lower limit L _b (=0.011943) and the upper limit U _b (=0.009294) corresponding to m/m'=1 of the baseline scan response delay b _RTT1 in the data set S ₁ shown in FIG. 11, respectively. The equation in the second line means that C(a _2Lb ) and C(a _2Ub ) are calculated using the lower limit L b (=0.011944) and the upper limit U b (=0.009293) corresponding to m/m'=1 of the baseline scan response delay b _RTT2 , respectively. Similarly, the equation in the last line means that C(a _2Lb ) and C(a _2Ub ) are calculated using the lower limit L _b (=0.011944) and the upper limit U _b (=0.009293) corresponding to m/m'=1 of the baseline scan response delay b RTT2, respectively. This means that C( _aALb ) and C( _aAUb ) are calculated using the lower limit _Lb (=0.019396) and upper limit _Ub (=0.015091) corresponding to m/m'=8 of _RTT8 . In other words, equation (10) means that in the data set S1 shown in FIG. ₁₁ , the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m'=1, the baseline scan response delays _bRTT1 to _bRTT8 corresponding to m/m'=2, ..., the baseline scan response delays bRTT1 to _bRTT8 corresponding to m/m'= ₈ are sequentially substituted into equation (8B) to calculate C(x _{i _ bRTT} ). As a result, A consists of 8×8=64 (that is, [total number M (or M′) of terminal devices in each of the data sets S ₁ to S ₄ ]×[number of ports in one terminal device m/m′]).

After calculating the mean absolute deviations W _unknown , W _{known — 1} , W _{known — 2} , and W _{known — 3} , the calculation means 16 calculates the similarity ratio b _sim by the following equation.

In this case, the calculation means 16 calculates the similarity rate _bsim1 by substituting the mean absolute deviations _Wunknown and _{Wknown_1} into equation (11), calculates the similarity rate bsim2 by substituting the mean absolute deviations _Wunknown and _{Wknown_2} into equation (11), and calculates the similarity rate _bsim3 by substituting the mean absolute deviations _Wunknown and _{Wknown_3} into equation ( ₁₁ ).

Then, the calculation means 16 outputs the three calculated similarity rates b _sim1 , b _sim2 , and b _sim3 to the update means 14 .

The update means 14 receives three similarity rates b _sim1 , b _sim2 , and b _sim3 from the calculation means 16, and detects the largest similarity rate b _{sim_max} (any of b _sim1 , b _sim2 , and b _sim3 ) among the received three similarity rates b _sim1 , b _sim2 , and b _sim3 . If there are multiple largest similarity rates b _{sim_max} , the update means 14 detects any one of the multiple largest similarity rates b _{sim_max} .

Transforming equation (11) gives the following equation:

Fig. 25 is a diagram showing the relationship between the similarity rate _bsim and the mean absolute deviation _Wknown shown in formula (12). In Fig. 25, the vertical axis represents the similarity rate _bsim , and the horizontal axis represents the mean absolute deviation _Wknown . Also, curve k3 shows the relationship between the similarity rate _bsim and the mean absolute deviation _Wknown in formula (12) when the mean absolute deviation _Wunknown is a constant value (=3). Note that in Fig. 25, the mean absolute deviations _Wknown and _Wunknown are not calculated using the specific values of the baseline scan response delay _bRTT shown in Figs. 10, 11, 13, and 14, but are arbitrary values for easily understanding how the similarity rate _bsim changes with respect to the value of the mean absolute deviation _Wknown . Therefore, in FIG. 25, the mean absolute deviation W _unknown is set to a constant value (for example, "3"), and the mean absolute deviation W _known is changed within the range of 1-11.

25, the similarity ratio _bsim is maximum and is 1.0 when _Wknown = _Wunknown = 3. Also, the similarity ratio _bsim becomes smaller than the maximum value (= 1.0) when the mean absolute deviation _Wknown shifts to a smaller side than the mean absolute deviation _Wunknown , or when the mean absolute deviation _Wknown shifts to a larger side than the mean absolute deviation _Wunknown (see curve k3). Therefore, the similarity ratio _bsim increases as the mean absolute deviation _Wknown approaches the mean absolute deviation _Wunknown .

Fig. 26 is a diagram for explaining a method for detecting the maximum similarity rate b _{sim_max} . Referring to Fig. 26, when the similarity rate b sim_1 is _the maximum among the three similarity rates b _{sim_1} , b _{sim_2} , and b _{sim_3} , the maximum similarity rate b _{sim_max} is made up of the similarity rate b _{sim_1} , when the similarity rate b _{sim_2} is the maximum, the maximum similarity rate b _{sim_max} is made up of the similarity rate b _{sim_2} , and when the similarity rate b _{sim_3} is the maximum, the maximum similarity rate b _{sim_max} is made up of the similarity rate b _{sim_3} .

In addition, for three similarity rates b _{sim_1} , b _{sim_2} , and b _{sim_3} , when b _{sim_1} = b _{sim_2} > b _{sim_3} , the maximum similarity rate b _{sim_max} is made up of the similarity rate b _{sim_1} or the similarity rate b _{sim_2} , when b _{sim_1} = b _{sim_3} > b _{sim_2} , the maximum similarity rate b _{sim_max} is made up of the similarity rate b _{sim_1} or the similarity rate b _{sim_3} , when b _{sim_2} = b _{sim_3} > b _{sim_1} , the maximum similarity rate b _{sim_max} is made up of the similarity rate b _{sim_2} or the similarity rate b _{sim_3} , and when b _{sim_2} = b _{sim_3} > b _{sim_1} , the maximum similarity rate b _{sim_max} is made up of the similarity rate b It consists of any one of _{a sim_1} , b _{sim_2} , and b _{sim_3} .

When detecting the maximum similarity rate b _{sim_max} using the method described in FIG. 26, the update means 14 receives three similarity rates b _{sim_1} , b _{sim_2} , and b _{sim_3} from the calculation means 16, and detects the maximum similarity rate b _{sim_max} using the method described above based on the received b _{sim_1} , b _{sim_2} , and b _{sim_3} .

When the update means 14 detects the largest similarity rate _{bsim_max} , the update means 14 updates the scan response delay { _RTT m _' ,PNr _known_bsim_max } (m'=1' to M') (scan response delay { _{RTT m',PNr} _known_bsim_max } (m'=1' to M') that was used to calculate the baseline scan response delay bRTT (any of baseline scan response delay { _{bRTT m',PNr} _known}_1 (m'=1' to M'), { _{bRTT m' ,PNr} _known}_2 (m'=1' to M'), {bRTT _m',PNr _known} _{_3} (m'=1' to M')) when the largest similarity rate bsim_max is obtained. _known}_1 (m' = 1' to M'), {RTT _{m', PNr} _known}_2 (m' = 1' to M'), {RTT _{m', PNr} _known}_3 (m' = 1' to M')) and the scan response delay {RTT _{m, PNr} _unknown} (m = 1 to M) to generate a scan response delay {RTT _{m', PNr} _known_aggregation} (m' = 1' to M').

A method of aggregating the scan response delay {RTT _{m', PNr} — known_b _{sim — max} } (m'=1' to M') and the scan response delay {RTT _{m, PNr —} unknown} (m=1 to M) will be described.

27 is a diagram for explaining a method of aggregating two similar sets. With reference to (a) of FIG. 27, sets S _A and S _B are two sets when the largest similarity rate b _{sim_max} is obtained. Each of sets S _A and S _B includes 16 scan response delays RTTs of one terminal device as elements.

The update means 14 detects a set S _A ∩ S _B of scan response delay RTTs included in both the set S _A and the set S _B (see FIG. 27B). In this case, the set S _A includes four "0.019", six "0.021", and two "0.020999", but when detecting the set S _A ∩ S _B , the four "0.019", six "0.021", and two "0.020999" are respectively converted into one "0.019", one "0.021", and one "0.020999" to form the set S _A ∩ S _B.

Then, the updating means 14 deletes the elements of the set S _A ∩S _B from the elements of the set S _B to generate a set S' _B (see FIG. 27(c)).

Then, the update means 14 adds the elements of the set _S'B to the elements of the set S _A to generate an aggregated set S _aggregation (see FIG. 27(d)). In FIG. 27(d), the set surrounded by the dotted line is the set _S'B . As a result, the set S _aggregation consists of 22 scan response delays RTT.

The set S _aggregation shown in FIG. 27(d) is an aggregation of the scan response delay RTT in one terminal device included in the set S _A and the scan response delay RTT in one terminal device included in the set S _B.

27 for the scan response delays RTTs of all terminal devices included in sets S _A and S _B , and aggregates the scan response delays RTTs included in set S _A and the scan response delays RTTs included in set S _B to generate aggregated scan response delays {RTT _{m', PNr} _known_aggregation} (m'=1' to M'). In the scan response delays {RTT _{m', PNr} _known_aggregation} (m'=1' to M'), the number of scan response delays RTTs in one terminal device m' may increase compared to the original set S _A.

FIG. 28 is a diagram for explaining a method of updating the known scan response delay {RTT _m' _known}_z (m'=1' to M', z=1 to 3).

Referring to Figure 28, in correspondence table TBL3, as shown in Figure 3, the scan response delay RTT is associated with a known estimation result (see (a) of Figure 28).

When a scan response delay {RTT _m _known} (m = 1 to M) is acquired in a communication environment where the result of a network scan is unknown, and the scan response delay {RTT _m' _known} _2 (m' = 1 to M') has the largest similarity rate b _{sim_max} with the scan response delay {RTT _m ' _known} _2 (m' = 1 to M), the update means 14 aggregates the scan response delay {RTT _m' _known} _2 (m' = 1 to M') and the scan response delay {RTT _m _known} (m = 1 to M) to generate a scan response delay {RTT _m' _known_aggregation} _2 (m' = 1 to M') by the above-mentioned method. Then, the update means 14 updates the correspondence table TBL3 by deleting the scan response delay {RTT _m' _known}_2 (m' = 1' to M') from the correspondence table TBL3 and storing the scan response delay {RTT _m' _known_aggregation}_2 (m' = 1' to M') in correspondence table TBL3 in association with the known estimation result of "insufficient communication quality" (see (a) and (b) of Figure 28).

Furthermore, when a new scan response delay {RTT _m _known} (m = 1 to M) is acquired in a communication environment where the result of a network scan is unknown, and the scan response delay {RTT _m' _known}_1 (m' = 1' to M') has the largest similarity rate b _{sim_max} with the new scan response delay {RTT _m _known} (m = 1 to M), the update means 14 aggregates the scan response delay {RTT _m' _known}_1 (m' = 1' to M') and the new scan response delay {RTT _m _known} (m = 1 to M) by the above-mentioned method to generate a scan response delay {RTT _m' _known_aggregation}_1 (m' = 1' to M'). Then, the update means 14 updates the correspondence table TBL3 by deleting the scan response delay {RTT _m' _known}_1 (m' = 1' to M') from the correspondence table TBL3 and storing the scan response delay {RTT _m' _known_aggregation}_1 (m' = 1' to M') in correspondence table TBL3 in association with the known estimation result of "normal" (see (b) and (c) of Figure 28).

In this way, each time a scan response delay {RTT _m _unknown} (m=1 to M) is acquired in a communication environment where the result of a network scan is unknown, the update means 14 aggregates the scan response delay {RTT _m _unknown} (m=1 to M) and the scan response delay {RTT _{m' _known} }_z (m'=1 to M', z=1 to 3) associated with a known estimation result to generate a scan response delay {RTT _m' _known_aggregation}_z (m'=1 to M', z=1 to 3), and calculates the scan response delay {RTT _m' _known}_z (m'=1' to M', z=1 to 3) is updated (that is, the host information database 13 is updated).

Although not shown in FIG. 28, the updated scan response delay {RTT _m' _known_aggregation}_2 (m' = 1' to M') may be further updated by the newly aggregated scan response delay {RTT _m' _known_aggregation(new)}_2 (m' = 1' to M').

In this way, in the scan response delays {RTT _m' _known}_z (m' = 1' to M', z = 1 to 3) associated with the known estimation results, the number of scan response delays RTT _m' acquired for one terminal device m' increases by aggregation. As a result, as shown in Fig. 92 described later, the probability density function of the scan response delay RTT when the known estimation result is "normal", the probability density function of the scan response delay RTT when the known estimation result is "inadequate communication quality", and the probability density function of the scan response delay RTT when the known estimation result is "network congestion" are mutually different. Therefore, it is possible to suppress errors when detecting, by the similarity rate b sim, a scan response delay {RTT _m' _known}_z (m' = 1' to M', z = 1 to 3) similar to a scan response delay {RTT _m _unknown}_z (m' = 1' to M', z = 1 to 3) obtained in a communication environment where the results of a network scan are unknown, among the scan response delays {RTT _m' _known}_z (m' = 1' to M _' , z = 1 to 3).

In this embodiment of the present invention, the aggregated scan response delays {RTT _m' _known_aggregation}_ 1 (where m'=1' to M') and {RTT _m' _known_aggregation}_ 2 (where m'=1' to M') constitute "M' aggregate scan response delays."

The similarity ratio b _sim calculated using the three types of scores z _{scr — c} , z _{scr — 1} to z _{scr — R} , and z _β will be described.

As described in Table 1, the baseline scan response delay _bRTT is calculated using one of the three scores _{zscr_c} , _{zscr_1} to _{zscr_R} , and _zβ . As a result, three types of baseline scan response delays _bRTT are calculated according to the three scores _{zscr_c} , _{zscr_1} to _{zscr_R} , and _zβ .

FIG. 29 is a diagram showing the similarity ratio b _sim calculated using three types of scores z _{scr — c} , z _{scr — 1} to z _{scr — R} , and z _β .

In Figure 29, (a) shows the similarity rate b _sim calculated using score z _{scr_c} for known estimation results (normal, insufficient communication quality, and network congestion), (b) shows the similarity rate b _sim calculated using scores z _{scr_1} to z _{scr_R} for known estimation results (normal, insufficient communication quality, and network congestion), and (c) shows the similarity rate b _sim calculated using score z _β for known estimation results (normal, insufficient communication quality, and network congestion).

As shown in (a), (b), and (c) of Fig. 29, the similarity rate _bsim calculated using the three scores _{zscr_c} , _{zscr_1} to _{zscr_R} , and _zβ is almost the same for the known estimation results (normal, insufficient communication quality, and network congestion). Therefore, the mean absolute deviations _{Wknown_1} , _{Wknown_2} , and _{Wknown_3} when the known estimation results are "normal", "inadequate communication quality", and "network congestion", respectively, are almost the same regardless of which of the three scores _{zscr_c} , _{zscr_1} to _{zscr_R} , and _zβ is used to calculate them, and the maximum similarity rate _{bsim_max} is the same regardless of which of the methods described in (a), (b), and (c) of Fig. 26 is used to detect the maximum similarity rate _{bsim_max} .

The advantage of aggregating scan response delays (RTTs) will now be described. a set _SC consisting of scan response delays {RTT _m' _known} _1 (m' = 1' to M') when the result of the network scan is the known estimated result of "normal", a set SD consisting of scan response delays {RTT _m' _known} _2 (m' = 1' to M') when the result of the network scan is the known estimated result of "inadequate communication quality", a set _SE consisting of scan response delays {RTT _m' _known} _3 (m' = 1' to M') when the result of the network _scan is the known estimated result of "network congestion", a set _SF consisting of scan response delays {RTT _m _unknown} _1 (m = 1 to M) acquired in a communication environment where the result of the network scan is unknown, and a set S consisting of scan response delays {RTT _m _unknown} _2 (m = 1 to M) acquired in a communication environment where the result of the network scan is unknown Assume that there exist sets S _C , S _D , and S _E. Here, the sets S C , S D , and S _E are made up of multiple types of scan response delays RTT that are dissimilar to each other. That is, the scan response delays {RTT _m' _known} _1 (m'=1' to M'), the scan response delays {RTT _m' _known} _2 (m'=1' to M'), and the scan response delays {RTT _m' _known} _3 (m'=1' to M') are dissimilar to each other.

FIG. 30 is a diagram for explaining a problem that occurs when determining whether a set S _G is a similar set.

With reference to FIG. 30, set S _E is similar to set S _F , set S _E is not similar to set S _G , and set S _F is similar to set S _G.

In this situation, the set S _F has a similarity rate b _sim with the set S _E of 0.89, that is, the largest similarity rate b _{sim_max} among the sets S _C , S _D , and S _E (see FIG. 30A ).

On the other hand, the set S _G has a similarity rate b _sim with the set S _D of 0.82, which is the largest similarity rate b _{sim_max} among the sets S _C , S _D , and S _E (see FIG. 30B ).

As a result, the probability that the set S _F is correctly estimated to be similar to the set S _E is 99.1% (see FIG. 30C), but the probability that the set S _G erroneously estimates a similar set is 98% (see FIG. 30D). Since the set S _G is similar to the set S _F , the similarity rate b _sim with the set S _E should be the largest, but in reality, the similarity rate b _sim with the set S _D is the largest. As a result, the probability that the set S _G erroneously estimates a similar set is 98%.

Therefore, in the assumed situation shown in FIG. 30, the result of the network scan when the scan response delay {RTT _m _unknown} — 2 (m=1 to M) of the set S _G is obtained may be erroneously determined.

FIG. 31 is a diagram showing the distribution of scan response delays RTT and baseline scan response delays _bRTT of sets S _E , S _F , and S _G .

In FIG. 31, (a) shows the distribution of scan response delays RTT and baseline scan response delays _bRTT of set S _E , (b) shows the distribution of scan response delays RTT and baseline scan response delays _bRTT of set S _F , and (c) shows the distribution of scan response delays RTT and baseline scan response delays _bRTT of set S _G.

Referring to FIG. 31, in the sets S _E , S _F , and S _G , the distribution range of the baseline scan response delay b _RTT is narrower than the distribution range of the scan response delay RTT.

Fig. 32 is a diagram showing the similarity rate of two sets, where (a) of Fig. 32 shows the similarity rate _bsim between the sets S _C , S _D , S _E and the set S _F , (b) of Fig. 32 shows the similarity rate _bsim between the sets S _C , S _D , S _E and the set S _G , and (c) of Fig. 32 shows the similarity rate _bsim between the sets S _C , S _D , S _F and the set S _G.

In order to identify two scan response delays {RTT _m _known}_1 (m=1 to M), {RTT _m _known}_2 (m=1 to M) when the result of a network scan is unknown but similar to the same scan response delay {RTT m' _known}_z (m'=1' to M', z=1 to 3) when a known estimation result is obtained, the scan response delay {RTT _{m '} _known}_z (m'=1' to M', z=1 to 3) when a known estimation result having a similarity rate b _sim larger than a threshold b _{sim_th} is obtained, and the scan response delay {RTT _m _known}_1 (m=1 to M) when the result of the network scan is unknown are aggregated. The threshold b _{sim_th} is set to, for example, 0.85.

As described above, the set S _C consists of the scan response delays {RTT _m' _known} _1 (m' = 1' to M'), the set S _D consists of the scan response delays {RTT _m' _known} _2 (m' = 1' to M'), the set S _E consists of the scan response delays {RTT _{m '} _known} _3 (m' = 1' to M'), the set S _F consists of the scan response delays {RTT m _unknown} _1 (m = 1 to M), and the set S _G consists of the scan response delays {RTT _m _unknown} _2 (m = 1 to M). Therefore, (a) of FIG. 32 shows the scan response delays {RTT _m' 32(b) shows the similarity rate b sim between the scan response delay {RTT _m' _known} _1 (m'=1' to M'), scan response delay {RTT _m' _known} _2 (m'=1' to M'), and scan response delay {RTT _m' _known} _3 (m'=1' to M') and the scan response delay {RTT m' _unknown} _1 (m=1 to M) obtained when the result of the network scan is unknown, and FIG. 32(b) shows the similarity rate b _sim between the scan response delay {RTT _m' _known} _1 (m'=1' to M'), scan response delay {RTT _m' _known} _2 (m'=1' to M'), and scan response delay {RTT _m' 32(c) shows the similarity rate _bsim between the scan response delay {RTT m' _known}_1 (m'=1' to M'), scan response delay {RTT _m' _known}_2 (m'=1' to M') when a known estimated result is obtained, and the similarity rate bsim between the scan response delay {RTT _m' _known}_1 (m'=1' to M') and the scan response delay {RTT _m' _known}_2 (m'=1' to M') obtained when the result of the network scan is unknown, and the scan response delay {RTT _m _known}_1 (m=1 to M) obtained when the result of the network scan is unknown, and the scan response delay {RTT _m _known}_2 (m=1 to M) obtained when the result of the network _scan is unknown.

In FIG. 32A, the similarity rate b _sim between the sets S _E and S _F is the maximum similarity rate b _{sim — max} (=0.89) which is greater than the threshold value b _{sim — th} (=0.85), so the set S _E is similar to the set S _F.

In FIG. 32B, the similarity rate b _sim between the sets S _E and S _G is "0.73" which is smaller than the threshold value b _{sim_th} (=0.85), so the set S _E is not similar to the set S _G.

In FIG. 32C, the similarity rate b _sim between the sets S _F and S _G is the maximum similarity rate b _{sim — max} (=0.94) which is greater than the threshold value b _{sim — th} (=0.85), so the set S _F is similar to the set S _G.

Therefore, the similarity/dissimilarity relationship between the sets S _E , S _F , and S _G described in the assumed situation in FIG. 30 is established.

Then, when the similarity/dissimilarity relationship between the sets S _E , S _F , and S _G described in the assumed situation in FIG. 30 is established, there is a problem in that a set similar to the set S _G is erroneously estimated, as described above.

32(a) and 32(b), the updating means 14 detects two sets S _E and S _F having a similarity rate b _sim (=0.89) greater than the threshold value b _sim — th (=0.85), and aggregates the two detected sets S _E and S _F by the above-mentioned method. That is, the updating means 14 aggregates the scan response delay {RTT _m' _known}_3 (m'=1' to M') when a known estimation result (network congestion) is obtained and the scan response delay {RTT _m _unknown}_1 (m=1 to M) obtained when the result of the network scan is unknown by the above-mentioned method.

Fig. 33 is a diagram showing examples of scan response delays {RTT _m' _known}_3 (m'=1' to M') and {RTT _m _unknown}_1 (m=1 to M) of sets S _E and _{S F.} (a) of Fig. 33 shows an example of scan response delays {RTT _m' _known}_3 (m'=1' to M') of set S _E , and (b) of Fig. 33 shows an example of scan response delays {RTT _m _unknown}_1 (m=1 to M) of set S F. In each of sets S _E and S _F , the number of ports is 16, and the number of terminal devices m/m' is 8.

When aggregating two sets S _E and S _F , the update means 14 detects 16 scan response delays RTT _E corresponding to m/m' = 1 from the set S _E , detects 16 scan response delays RTT _F corresponding to m/m' = 1 from the set S _F , and aggregates the detected 16 scan response delays RTT _E and 16 scan response delays RTT _F using the method described above.

Then, the update means 14 detects 16 scan response delays RTT _E corresponding to m/m' = 2 from the set S _E , detects 16 scan response delays RTT F corresponding to m/m' = 2 from the set S _F , and aggregates the detected _{16 scan response delays RTT E and 16 scan response delays RTT F by the} above-mentioned method.

Similarly, the updating means 14 detects 16 scan response delays RTT _E from the set S _E , detects 16 scan response delays RTT _F from the set S _F , and aggregates the detected 16 scan response delays RTT _E and 16 scan response delays RTT _F using the above-mentioned method for all six sets of 16 scan response delays RTT _E and 16 scan response delays RTT _F corresponding to m/m' = 3 to 8, and aggregates the two sets S _E and S _F.

34 is a diagram showing a set S _H that is an aggregate of sets S _E and S _F. Referring to FIG 34, set S _H is composed of scan response delays {RTT _m' _known}_3 (m'=1' to M') of set S _E and scan response delays RTT of set S _F that are not common between sets S _E and S _F.

The scan response delay {RTT _m' _known}_3 (m'=1' to M') of the set S _E consists of 16 scan response delays RTT corresponding to each of m/m'=1 to 8. In addition, the scan response delay RTT of the set S _F not common between the sets S _E and S _F consists of one or more scan response delays RTT of the set S _F not common between the sets S _E and S _F corresponding to each of m/m'=1 to 8.

As a result, the set S _H includes 16 scan response delays RTTs corresponding to each of m/m'=1 to 8, and one or more scan response delays RTTs in the set S _F that are not common between the sets S _E and S _F .

FIG. 35 is a diagram showing an example of scan response delays {RTT _m' _known}_3 (m'=1' to M') of the set S _E shown in FIG. 34 and an example of scan response delays _RTT of the set S _F that are not common between the sets S E and S _F.

FIG. 35(a) shows an example of a scan response delay {RTT _m' _known}_3 (m' = 1' to M') of the set S _E shown in FIG. 34, and FIG. 35(b) shows an example of a scan response delay RTT of the set S _F that is not common between the sets S _E and S _F shown in FIG. 34.

Referring to FIG. 35 , six scan response delays RTTs corresponding to m/m′=1 of the scan response delays RTTs of set S _F that are not common between sets S _E and S _F are added to 16 scan response delays _RTTs corresponding to m/m′=1 of the scan response delays {RTT _m′ _known}_3 (m′=1′ to M′) of set S E.

Furthermore, 16 scan response delays RTTs corresponding to m/m'=2 of the scan response delays RTTs of set S _F that are not common between sets S _{E and} S _F are added to the 16 scan response delays RTTs corresponding to m/m'=2 of the scan response delays {RTT _m' _known}_3 (m'=1' to M') of set S E. The same applies to the scan response delays RTTs corresponding to m/m'=3 to 8.

Fig. 36 is a diagram showing the scan response delay {RTT _m _unknown}_2 (m=1 to M) of the set S _G. Fig. 37 is a diagram showing the baseline scan response delay of the sets S _G and S _H.

FIG. 37(a) shows the baseline scan response delays _b RTT of eight terminal devices calculated by randomly selecting one _scan response delay RTT from the 16 scan response delays RTT corresponding to each terminal device m/m′ in the scan response delays {RTT m _unknown} _2 (m = 1 to M) of the set S _G shown in FIG. 36 .

In addition, (b) of Figure 37 shows the baseline scan response delays b _RTT of eight terminal devices calculated by randomly selecting one scan response delay RTT from the multiple scan response delays RTT corresponding to each terminal device m/m' in the scan response delays of the set _{S H} shown in Figures 34 and 35.

FIG. 38 is a diagram showing the distribution of scan response delays RTT of sets S _G and S _H and the baseline scan response delays b _RTT .

In FIG. 38, (a) shows the distribution of the scan response delay RTT and the baseline scan response delay _bRTT of the set S _G , and (b) shows the distribution of the scan response delay RTT and the baseline scan response delay _bRTT of the set S _H.

In both sets S _G and S _H , the distribution range of the baseline scan response delay b _RTT is narrower than the distribution range of the scan response delay RTT.

Fig. 39 is a diagram showing the similarity rate _bsim between the sets S _C , S _D , and S _H and the set S _G. The similarity rate _bsim shown in Fig. 39 was calculated using the baseline scan response delay _bRTT shown in Fig. 37. Referring to Fig. 39, the similarity rate _bsim between the sets S _C and S _G and the similarity rate _bsim between the sets S _D and S _G are smaller than the threshold value _{bsim_th} (=0.85). On the other hand, the similarity rate _bsim between the sets S _H and S _G is "1.00", which is larger than the threshold value _{bsim_th} (=0.85). Therefore, the set S _G is not similar to the sets S _C and S _D , but is similar to the set S _H.

Figure 40 shows a comparison between a method that aggregates scan response delays and a method that does not aggregate scan response delays.

Referring to FIG. 40, when a method that does not aggregate scan response delays is used, the scan response delay RTT of the set S _G has the largest similarity rate b _sim with the scan response delay RTT when the known estimation result is “insufficient communication quality (Low-SNR)” (see (a) of FIG. 40).

As a result, there is a 98% probability that the set S _G will erroneously estimate a similar set (see FIG. 40(b)).

On the other hand, when the method of aggregating scan response delays is used, the scan response delay RTT of set S _G has the largest similarity rate b _sim with the scan response delay RTT (= the scan response delay RTT of set S _E ) when the known estimation result is “network congestion” (see (c) of FIG. 40 ).

As a result, the set S _G correctly estimates similar sets with a probability of 99% (see FIG. 40(d)).

The set S _H is composed of a set S E consisting of scan response delays {RTT _m' _known} _ 3 (m' = 1' to M') when the known estimation result is "network congestion" and a set S _F (set S _F is similar to set S _E ) consisting of scan response delays {RTT _m _unknown} _ ₁ (m = 1 to M) when the result of the network scan is unknown.

More specifically, set _SH is composed of the scan response delays {RTT _m _known} _3 (m' = 1' to M') of set _SE plus, for each terminal _device m/ _{m', the scan response delays RTT m _unknown} _1 (m = 1 to M) of set SF that are not included in set _SE (see Figures 34 and 35).

As a result, since the set S _H contains a set S _F that is similar to the set S _G , the set S _G is correctly estimated to be similar to the set S _H consisting of scan response delays when the known estimated result is “network congestion” (or similar).

Therefore, by using the method of aggregating scan response delays, the problem described in FIG. 30 that "the sets S _G erroneously estimate similar sets with a high probability" can be solved.

As described above, in the embodiment of the present invention, after estimating the result of a network scan, the scan response delay {RTT _m' _known}_similarity (m'=1' to M') that is most similar to the scan response delay {RTT _m' _known}_(m'=1' to M') is detected from among the scan response delays {RTT _m' _known}_1 (m'=1' to M'), {RTT _m' _known}_2 (m'=1' to M'), and {RTT _m' _known}_3 (m'=1' to M') when a known estimated result of the network scan is obtained, and the detected scan response delay {RTT m _' _known}_similarity (m'=1' to M') and the scan response delay {RTT _m _unknown} (m=1 to M), and updates the correspondence table TBL3 with the aggregated scan response delay {RTT _m' _known_aggregation} (m'=1 to M') (that is, updates the host information database 13). This makes it possible to solve the problem of erroneously estimating the scan response delay {RTT _m' _known_similarity}_z (m' = 1' to M', z = 1 to 3) that is similar to the scan response delay _{{RTT m _unknown} } (m = 1 to M) among the scan response delays {RTT _m' _known}_1 (m' = 1' to M'), {RTT _m' _known}_2 (m' = 1' to M'), and {RTT m' _known}_3 (m' = 1' to M').

Next, we explain how to extract the communication characteristics CMbest that are most suitable for estimating the results of a network scan.

Fig. 41 is a diagram showing correlation coefficients of communication characteristics. The correlation coefficients shown in Fig. 41 are correlation coefficients of communication characteristics when communication quality is insufficient when a network scan is performed without monitoring background traffic. The scan response delay RTT _m , packet loss rate PLR _m , scan maximum delay DLmax _m , and scan minimum delay DLmin _m shown in Fig. 41 are multiple communication characteristics of M terminal devices when a network scan is performed on M terminal devices. The scan response delay RTT _m indicates one scan response delay or an average value of multiple scan response delays within one IP address that is the target of a network scan.

The correlation coefficients between the scan response delay RTT _m , packet loss rate PLR _m , scan maximum delay DLmax _m , and scan minimum delay DLmin _m are shown in Fig. 41. Note that in Fig. 41, the correlation coefficients of the off-diagonal elements above the diagonal elements are not shown because they are the same as the correlation coefficients of the off-diagonal elements below the diagonal elements.

41 , when a network scan is performed without monitoring background traffic and communication quality is insufficient, the correlation coefficient between the scan response delay RTT _m and the scan maximum delay DLmax _m , the correlation coefficient between the scan response delay RTT _m and the scan minimum delay DLmin _m , and the correlation coefficient between the scan maximum delay DLmax _m and the scan minimum delay DLmin _m are greater than 0.5. Also, the correlation coefficient between the scan response delay RTT _m and the scan maximum delay DLmax _m , and the correlation coefficient between the scan response delay RTT _m and the scan minimum delay DLmin _m are greater than the correlation coefficient between the scan maximum delay DLmax _m and the scan minimum delay DLmin _m . Therefore, the scan response delay RTT _m has a large correlation with the scan maximum delay DLmax _m and the scan minimum delay DLmin _m .

FIG. 42 is a diagram showing another correlation coefficient of communication characteristics. The correlation coefficient shown in FIG. 42 is a correlation coefficient of communication characteristics when a network scan is performed without monitoring background traffic and the network is congested (network congestion). The scan response delay RTT _m indicates an average value within one IP address that is a target of a network scan. The mutual correlation coefficients of the scan response delay RTT _m , the packet loss rate PLR _m , the scan maximum delay DLmax _m , and the scan minimum delay DLmin _m are shown in FIG. 42. Note that the reason why the correlation coefficients of the off-diagonal elements above the diagonal elements are not shown in FIG. 42 is the same as that explained in FIG. 41. The scan response delay RTT _m , the packet loss rate PLR _m , the scan maximum delay DLmax _m , and the scan minimum delay DLmin _m shown in FIG. 42 are a plurality of communication characteristics of M terminal devices when a network scan is performed on M terminal devices.

42, in the case where a network scan is performed without monitoring background traffic, when the network is congested, the correlation coefficient between the scan response delay RTT _m and the scan maximum delay DLmax _m , the correlation coefficient between the scan response delay RTT _m and the scan minimum delay DLmin _m , and the correlation coefficient between the scan maximum delay DLmax _m and the scan minimum delay DLmin _m are greater than 0.5. The correlation coefficient between the scan response delay RTT _m and the scan maximum delay DLmax _m and the correlation coefficient between the scan response delay RTT _m and the scan minimum delay DLmin _m are the same. Also, the correlation coefficient between the scan response delay RTT _m and the scan maximum delay DLmax _m and the correlation coefficient between the scan response delay RTT _m and the scan minimum delay DLmin _m are greater than the correlation coefficient between the scan maximum delay DLmax _m and the scan minimum delay DLmin _m . Therefore, even when the network is congested, the scan response delay RTT _m has a large correlation with the scan maximum delay DLmax _m and the scan minimum delay DLmin _m .

As shown in Figures 41 and 42, when a network scan is performed without monitoring background traffic, the correlation coefficients between the scan response delay RTT _m , packet loss rate PLR _m , scan maximum delay DLmax _m , and scan minimum delay DLmin _m differ when the communication quality is insufficient and when the network is congested.

Figure 43 is a diagram showing yet another correlation coefficient of communication characteristics. Note that the correlation coefficient shown in Figure 43 is the correlation coefficient of communication characteristics when communication quality is insufficient when a network scan is performed by monitoring background traffic.

The correlation coefficients between the scan response delay RTT _m , packet loss rate PLR _m , delay DL _m , jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m are shown in Figure 43. Note that the correlation coefficients of the off-diagonal elements above the diagonal elements are not shown in Figure 43 for the same reason as explained in Figure 41. The scan response delay RTT _m , packet loss rate PLR _m , delay DL _m , jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m shown in Figure 43 are multiple communication characteristics of M terminal devices when a network scan is performed on M terminal devices.

Referring to FIG. 43 , when a network scan is performed by monitoring background traffic and communication quality is insufficient, the correlation coefficient between the scan response delay RTT _m and the packet loss rate PLR _m , delay DL _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m , the correlation coefficient between the packet loss rate PLR _m and the delay DL _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m , the correlation coefficient between the delay DL _m and jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m , the correlation coefficient between the jitter JT _m and the background maximum delay BGDLmax _m and background minimum delay BGDLmin _m , and the correlation coefficient between the background maximum delay BGDLmax _m and background minimum delay BGDLmin _m are greater than 0.5. The scan response delay RTT _m has the largest correlation coefficient with the packet loss rate PLR _m .

Figure 44 is a diagram showing yet another correlation coefficient of communication characteristics. Note that the correlation coefficient shown in Figure 44 is the correlation coefficient of communication characteristics when the network is congested when a network scan is performed by monitoring background traffic.

The correlation coefficients between scan response delay RTT _m , packet loss rate PLR _m , delay DL _m , jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m are shown in Figure 44. Note that the correlation coefficients of off-diagonal elements above the diagonal elements are not shown in Figure 44 for the same reason as explained in Figure 41. The scan response delay RTT _m , packet loss rate PLR _m , delay DL _m , jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m shown in Figure 44 are multiple communication characteristics of M terminal devices when a network scan is performed on M terminal devices.

Referring to FIG. 44, when a network scan is performed by monitoring background traffic, when the network is congested, the correlation coefficient between the scan response delay RTT _m and the packet loss rate PLR _m , delay DL _m , jitter JT _m , and background maximum delay BGDLmax _m , the correlation coefficient between the packet loss rate PLR _m and the delay DL _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m , the correlation coefficient between the delay DL _m and jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m , the correlation coefficient between jitter JT _m and the background maximum delay BGDLmax _m and background minimum delay BGDLmin _m , and the correlation coefficient between the background maximum delay BGDLmax _m and background minimum delay BGDLmin _m are greater than 0.5. And the scan response delay RTT _m has the largest correlation coefficient with the background maximum delay BGDLmax _m .

As shown in Figures 43 and 44, when a network scan is performed by monitoring background traffic, the mutual correlation coefficients of the scan response delay RTT _m , packet loss rate PLR _m , delay DL _m , jitter JT _m , background maximum delay BGDLmax _m , and background minimum delay BGDLmin _m differ when communication quality is insufficient and when the network is congested.

In addition, the correlation coefficient between the scan response delay RTT _m shown in Figures 41 to 44 and the packet loss rate PLR _m , delay DL _m , jitter JT _m , maximum background delay BGDLmax _m , and minimum background delay BGDLmin _m may be a correlation coefficient calculated by using the scan response delay RTT _m stored in the correspondence table TBL1 shown in Figure 3 or the correspondence table TBL2 shown in Figure 4, averaged by the number of ports at which there was a scan response SR _m in one terminal device, as the scan response delay RTT _m , or may be a correlation coefficient calculated by using each communication characteristic such as the scan response delay RTT _m and packet loss rate PLR _m stored in the correspondence tables TBL1 and TBL2 as is (i.e., with regard to the scan response delay RTT _m , taking into consideration the difference in the communication characteristics of the scan response delay RTT _m due to the difference in ports).

When a network scan is performed, the results of the network scan are classified as successful or failed. If the network scan fails, the following first, second, third, and fourth causes are assumed to be the reasons why the network scan failed.

(First cause)
The communication quality with the terminal device m that is the target of the network scan is insufficient.

(Second cause)
The network to which the terminal device m, which is the target of the network scan, is connected is congested (i.e., network congestion).

(Third cause)
The terminal device m that is the target of the network scan is not connected to the network.

(Fourth cause)
The port of the terminal device m that is the target of the network scan is blocked.

Therefore, in the embodiment of the present invention, the communication characteristics such as the scan response delay RTT _m , packet loss rate PLR _m , and delay DL _m obtained when a network scan is executed are used to estimate whether the network scan is normal or unsuccessful, and when it is estimated that the network scan has failed, the cause of the failure (any of the first, second, third, or fourth cause) is estimated. Note that the first and second causes are causes attributable to the communication channel with the terminal device.

The method for estimating the result of a network scan includes an extraction process for extracting a communication characteristic CMbest that is most suitable for estimating the result of a network scan from communication characteristics such as a scan response delay RTT _m , a packet loss rate PLR _m , and a delay DL _m shown in Figures 41 to 44, and an estimation process for classifying terminal devices having the communication characteristics extracted in the extraction process into a group CLS ₁ where the network scan is normal and groups CLS ₂ and CLS ₃ where the network scan has failed due to a cause caused by _the communication channel with the terminal _device , calculating a D-score D_2 for distinguishing between groups CLS ₁ and CLS ₃ , or a D-score D_3 for distinguishing between groups CLS 2 and CLS 3, estimating that the network scan is normal or unsuccessful based on the calculated D-score D_2 or D-score D_3, and estimating the cause of the failure of the network scan when it is estimated that the network scan has failed.

[Extraction of communication characteristic CMbest _m _unknown]
When a network scan is performed without monitoring background traffic according to the schedule SCH_scan shown in FIG. 6 , the scan analysis manager 12 receives the scan response delay RTT _m , packet loss rate PLR _m , scan maximum delay DLmax _m , and scan minimum delay DLmin _m at one scan timing _ts for each of the M terminal devices from the network scanner 11, and creates or updates the host information database 13 by storing the received scan response delay RTT _m , packet loss rate PLR _m , scan maximum delay DLmax _m , and scan minimum delay DLmin _m in the correspondence table TBL1 of the host information database 13.

The extraction means 15 reads out M scan response delays RTT _m , M packet loss rates PLR _m , M maximum scan delays DLmax _m and M minimum scan delays DLmin _m from the correspondence table TBL1 of the host information database 13, and calculates the mutual correlation coefficient r ij of the scan response delays RTT _m , packet loss rates PLR _m , M maximum scan delays DLmax _m and M minimum scan delays DLmin _m based on the read out M scan response delays RTT _m , M packet loss rates PLR _m , M maximum scan delays DLmax _m and M minimum scan delays DLmin _m using the following _formula :

In formula (13), x _ai represents a sample of one communication characteristic f _i from which the correlation coefficient r _ij is calculated, and y _aj represents a sample of the other communication characteristic f _j from which the correlation coefficient r _ij is calculated, where a _i =1 to M and a _j =1 to M.

The extraction means 15 extracts correlation coefficients r _ij (i≠j) of different communication characteristics f _i , f _j (i≠j) from among the correlation coefficients r _ij of the communication characteristics f _i , f _j . For example, when the calculated correlation coefficients r _ij are those shown in Fig. 41, the communication characteristics f _i (i=1 to 4) are set to the scan response delay RTT _m , packet loss rate PLR _m , scan maximum delay DLmax _m and scan minimum delay DLmin _m in the column direction, respectively, and the communication characteristics f _j (j=1 to 4) are set to the scan response delay RTT _m , packet loss rate PLR _m , scan maximum delay DLmax _m and scan minimum delay DLmin _m in the row direction, respectively.

The correlation coefficients _r11 , _r22 , _r33 , and _r44 are diagonal elements, and the correlation coefficients _r21 , _r31 , _r41 , _r32 , _r42 , and _r43 are off-diagonal elements.

The extraction means 15 sequentially extracts correlation coefficients _r21 , r31, _r41 , _r32 , _r42 , and _r43 of the off-diagonal elements, and judges whether each of the extracted correlation coefficients _r21 , _r31 , _r41 , _r32 , _r42 , and _r43 is greater than a threshold value Corr _r . _The threshold value Corr _r is set to a real number in the range of 0.5 to 1.0, for example. Here, the threshold value Corr _r = 0.5.

First, the extraction means 15 judges whether the correlation coefficient _r21 is greater than a threshold value _{Corr_r} . If the correlation coefficient _r21 is equal to or less than the threshold value _{Corr_r} , the extraction means 15 deletes the two communication characteristics _f2 and _f1 used to calculate the correlation coefficient _r21 , and if the correlation coefficient _r21 is greater than the threshold value _{Corr_r} , the extraction means 15 stores the two communication characteristics _f2 and _f1 used to calculate the correlation coefficient _r21 .

In this case, since the correlation coefficient _r21 (=0.25) is less than the threshold value _Corrr (=0.5), the extraction means 15 deletes the two communication characteristics _f2 (=packet loss rate _PLRm ) and _f1 (=scan response delay _RTTm ) that were used to calculate the correlation coefficient _r21 .

Next, the extraction means 15 judges whether the correlation coefficient _r31 is greater than the threshold value _Corr3r . Since the correlation coefficient _r31 is 0.80, the extraction means 15 judges that the correlation coefficient _r31 is greater than the threshold value _Corr3r , and saves the two communication characteristics _f3 (= maximum scan delay _DLmaxm ) and _f1 (= scan response delay _RTTm ) that were used to calculate the correlation coefficient _r31 .

Next, the extraction means 15 judges whether the correlation coefficient _r41 is greater than the threshold value Corr _r . Since the correlation coefficient _r41 is 0.97, the extraction means 15 judges that the correlation coefficient _r41 is greater than the threshold value Corr _r , and saves the two communication characteristics _f3 (=scan minimum delay DLmin _m ) and _f1 (=scan response delay RTT _m ) that were used to calculate the correlation coefficient _r41 .

Similarly, the extraction means 15 sequentially determines whether the correlation coefficients _r32 , _r42 , and _r43 are greater than the threshold value _Corrr . The extraction means 15 then stores the two communication characteristics _f4 (=minimum scanning delay _DLminm ) and _f3 (=maximum scanning delay DLmaxm) that were used to calculate the correlation coefficient _r43 that is greater than the threshold value _Corrr (=0.5). The extraction means 15 also deletes the two communication characteristics _f3 (=maximum scanning delay _DLmaxm ) and _f2 (=packet loss rate _PLRm ) that were used to calculate the correlation coefficients _r32 and _r42 that are less than the threshold value Corrr (= _0.5 ); _and the two communication characteristics _f4 (=minimum scanning delay _DLminm ) and _f2 (=packet loss rate _PLRm ).

As a result, two communication characteristics _f3 (= maximum scan delay DLmax _m ), _f1 (= scan response delay RTT _m ); two communication characteristics _f3 (= minimum scan delay DLmin _m ), _f1 (= scan response delay RTT _m ); and two communication characteristics _f4 (= minimum scan delay DLmin _m ), _f3 (= maximum scan delay DLmax _m ) are saved, so the extraction means 15 finally saves three communication characteristics: scan response delay RTT _m , maximum scan delay DLmax _m and minimum scan delay DLmin _m .

In addition, even when the correlation coefficient r _ij is a correlation coefficient shown in any one of Figures 42 to 44, the extraction means 15 similarly stores the communication characteristics used to calculate the correlation coefficient r _ij that is greater than the threshold value Corr _r , and deletes the communication characteristics used to calculate the correlation coefficient r _ij that is equal to or smaller than the threshold value Corr _r .

The scan response delay RTT _m represents the delay from sending a packet of test input to receiving a scan response in a normal network scan, relative to the RTT _normal , and therefore best represents the results of a network scan.

Therefore, the extraction means 15 extracts the maximum scan delay DLmax _m and the minimum scan delay DLmin _m as explanatory variables from the finally saved communication characteristics (in the above example, scan response delay RTT _m , maximum scan delay DLmax _m , and minimum scan delay DLmin _m ), and extracts the scan response delay RTT _m as a response variable, and performs multiple regression analysis to find the contribution rate (= ^R2 ) in the multiple regression analysis.

Thereafter, the extraction means 15 judges whether the contribution rate ^R2 is smaller than a threshold value _Sr. The threshold value _Sr is set to a real number in the range of 0.5 to 1.0, for example.

When the extraction means 15 determines that the contribution rate ^R2 is smaller than the threshold value _Sr , it sets the mode of the estimation algorithm ALG that estimates the results of the network scan to "uncorrelated," and when it determines that the contribution rate ^R2 is not smaller than the threshold value _Sr , it sets the mode of the estimation algorithm ALG to "correlated."

When the mode of the estimation algorithm ALG is set to "uncorrelated", it means that the communication characteristics finally saved are somewhat important. On the other hand, when the mode of the estimation algorithm ALG is set to "correlated", it means that the communication characteristics finally saved are very important.

Then, the extraction means 15 uses the machine learning algorithm "SelectKbest" to select the communication characteristic CMbest that is most suitable for estimating the results of the network scan from the K communication characteristics.

A method for selecting a communication characteristic CMbest will be described. Figure 45 is a diagram showing an example of the original data when selecting a communication characteristic CMbest. Figure 46 is a diagram for explaining a method for selecting a communication characteristic CMbest.

As multiple communication characteristics obtained from the scan response of a network scan, the packet loss rate PLR _m , jitter JT _m , scan maximum delay DLmax _m , scan minimum delay DLmin _m , and delay DL _m are shown in Fig. 45. In Fig. 45, the packet loss rate PLR _m , jitter JT _m , scan maximum delay DLmax _m , scan minimum delay DLmin _m , and delay DL _m are shown for eight terminal devices.

Then, the extraction means 15 selects, by the _above -mentioned method, for example, a packet loss rate PLR _m , a jitter JT _m , a scanning maximum delay DLmax _m , _and a scanning minimum delay DLmin _m as shown in FIG. 46 as communication characteristics having a correlation coefficient r ij greater than the threshold value Corr r.

Then, the extraction means 15 calculates the sum of each row and the sum of each column in FIG. 46, and calculates a grand total, which is the sum of the sum of each row and the sum of each column.

Next, the extraction means 15 calculates the expected value E _rc of row r and column c for all rows and all columns by the following formula:

In equation (14), e _r represents the sum of the rth row, e _c represents the sum of the cth column, and N _grand represents the grand total.

Fig. 47 is a diagram for explaining a method for calculating a score of a communication characteristic. Fig. 48 is a diagram showing the score of a communication characteristic. Note that Fig. 47 explains a method for calculating a score Q of a maximum scan delay DLmax _m .

47, the leftmost column is the sum and grand total of the rightmost column of FIG. 46, and the second leftmost column is a sample of the scan maximum delay DLmax _m shown in FIG.

The extraction means 15 sequentially substitutes the sums of each row shown in FIG. 48 (=0.049, 0.034, 0.023, 0.038, 0.022, 0.022, 0.021, 0.023) into e _r of formula (14), substitutes the sum (=0.232) of the c-th column shown in FIG. 47 into e _c of formula (14), and substitutes the grand total of FIG. 47 into N _grand of formula (14) to calculate the expected value E _rc .

Then, the extraction means 15 calculates the score Q using the following formula:

In equation (15), O is each value of the maximum scan delay DLmax _m shown in FIG. 47, and E is the expected value E _rc calculated by equation (14).

The score Q represents the magnitude of deviation of each sample O from the expected value E in one communication characteristic (e.g., maximum scan delay DLmax _m ). In equation (15), when each sample O is significantly deviated from the expected value E toward the larger side or when each sample O is significantly deviated from the expected value E toward the smaller side, the score Q is large, and when each sample O is slightly deviated from the expected value E toward the larger side or when each sample O is slightly deviated from the expected value E toward the smaller side. Thus, the score Q represents the characteristics of one communication characteristic (e.g., maximum scan delay DLmax _m ).

After calculating the expected value E, the extraction means 15 calculates O−E, then calculates (O−E) ² , and divides the calculated (O−E) ² by the expected value E to calculate (O−E) ² /E.

Then, the extraction unit 15 calculates the sum of (OE) ² /E for each row as the score Q of the scan maximum delay DLmax _m . That is, the extraction unit 15 calculates the score Q of the scan maximum delay DLmax _m by the formula (15).

The extraction means 15 also calculates the score Q by the above-mentioned method for the packet loss rate PLR _m , jitter JT _m , and scan minimum delay DLmin _m shown in Fig. 46. As a result, as shown in Fig. 48, the score Q for the packet loss rate PLR _m , jitter JT _m , scan maximum delay DLmax _m , and scan minimum delay DLmin _m is obtained.

Then, the extraction means 15 selects K (K is a positive integer) communication characteristics in descending order of score Q from the scores Q of the packet loss rate PLR _m , jitter JT _m , scan maximum delay DLmax _m , and scan minimum delay DLmin _m .

For example, when K=3, in the example shown in FIG. 48, the extracting means 15 selects a jitter JT _m , a scan minimum delay DLmin _m and a scan maximum delay DLmax _m .

After selecting K communication characteristics, the extraction means 15 selects a communication characteristic CMbest from the K communication characteristics by the following method.

First, the extraction means 15 classifies the K communication characteristics using classification algorithms such as LR (Linear Regression) and SVM (Support Vector Machine) into a group GP1 of communication characteristics that are important for estimating the results of a network scan and a group GP2 of communication characteristics that are not important for estimating the results of a network scan.

Here, we use a support vector machine (SVM-train) as the classification algorithm.

The SVM is one of the pattern recognition models that uses supervised learning and learns the boundary that most boldly divides the training data _X0 = [ _x01 , _x02 , ..., _x0M ] ^T (T represents transposition). In order to learn this boundary, a two- or more-dimensional space composed of two or more parameters is set.

As a result of learning using learning samples arranged in a two- or more-dimensional space, the boundary that most boldly divides the teacher data _X0 = [ _x01 , _x02 , ..., _x0M ] ^T is determined so as to maximize the distance (margin) between the boundary and the sample (data) closest to the boundary. This determined boundary is called a maximum margin hyperplane. Note that a specific method for determining this boundary is well known and will not be described here.

The extraction means 15 executes learning of a support vector machine (SVM-train) using teacher data X ₀ = [x ₀₁ , x ₀₂ , ..., x _0M ] ^T (T represents transposition), and classifies each of M sample vectors x _m (m = 1 to M) into one of groups GP1 and GP2 using the boundaries determined by the learning, so that each of the M sample vectors x _m (m = 1 to M) satisfies the following formula. Here, the sample vector x _m is x _m = [CM _1m , CM _2m , ..., CM _km , ..., CM _Km ]. That is, the sample vector x _m includes K communication characteristics CM _1m to CM _Km as elements. As a result, the data to be classified into one of groups GP1 and GP2 is expressed by formula (17).

In equation (17), communication characteristic CM _km (k=1 to K, m=1 to M) represents the k-th communication characteristic of terminal device m.

In equation (16), z is a weight vector and b is a predetermined constant.

Group GP1 is a group consisting of important communication characteristics for estimating the results of a network scan. Group GP2 is a group consisting of unimportant communication characteristics for estimating the results of a network scan. When sample vector x _m is classified into group GP1, y _m is "1", and when sample vector x _m is classified into group GP2, y _m is "-1".

The extraction means 15 calculates the weight vector z using equation (16) to determine the classification index α in the following equation.

In formula (18), X = [ _x1 , _x2 , ..., _xM ] ^T (T represents transposition), and y = [ _y1 , _y2 , ..., _yM ] ^T (T represents transposition).

When sample vector _xm is classified into group GP1 (i.e., when _ym is "1"), extraction means 15 calculates weight vector z by z· _xm -b=1 and sets the calculated weight vector z as classification index α. When sample vector _xm is classified into group GP2 (i.e., when _ym is "-1"), extraction means 15 calculates weight vector z by z· _xm -b=-1 and sets the calculated weight vector z as classification index α.

When the extraction means 15 determines the classification index α for the kth communication characteristic CM _km , it substitutes the determined classification index α (= α _k ), the sample vector x _km , and y _m (= “1” or “−1”) into the following equation to calculate a weight w _k indicating the importance of the communication characteristic for estimating the results of the network scan.

In formula (19), x _km represents the k-th communication characteristic sample CM _km of the sample vector x _m in the terminal device m. Therefore, the right side of formula (19) means that the weight w _k is calculated using M k-th communication characteristics CM _k1 , CM _k2 , ..., CM _km , ..., CM _kM shown in formula (17).

Thereafter, the extraction means 15 substitutes the weight w _k calculated by the formula (19) into the following formula to calculate the rank weight R _k for determining the rank.

The extraction means 15 calculates the rank weights R _k for all of the K communication characteristics by the above-mentioned method.

Fig. 49 is a diagram showing the calculation results of the rank weight _Rk . Referring to Fig. 49, the rank weight _Rk and the rank are associated with the scan maximum delay _DLmaxm , the jitter _JTm , the scan minimum delay _DLminm , and the packet loss rate _PLRm .

Then, the extraction means 15 determines the communication characteristic (here, the packet loss rate PLR _m ) with the lowest rank (i.e., with the smallest rank weight R _k ) as the communication characteristic to be deleted. Thereafter, the extraction means 15 deletes the samples of the communication characteristic from the sample vectors x ₁ to x _M , and repeats the calculation of equations (16), (18) to (20) and the determination of the communication characteristic to be deleted until all communication characteristics are deleted. Note that, if there are multiple communication characteristics with the smallest rank weight R _k , the extraction means 15 deletes any one of these communication characteristics. Also, if there are multiple communication characteristics with the smallest rank weight R _k , the extraction means 15 may determine the communication characteristic with the smallest correlation coefficient r _ij with the scan response delay RTT _m as the communication characteristic to be deleted.

Then, the extraction unit 15 detects the last deleted communication characteristic as the communication characteristic CMbest _m _unknown, combines the detected communication characteristic CMbest _m _unknown with the scan response delay RTT _m _unknown, and outputs the combined result to the estimation unit 17 .

In addition, when a network scan is performed by monitoring background traffic according to the schedule SCH_scan shown in Figure 6, the extraction means 15 selects the communication characteristic CMbest m _unknown by the above-mentioned method based on the M scan response delays RTT _m , M packet loss rates PLR _m , M delays DL m, M jitters JT _m , M background maximum delays BGDLmax _m , _and M background minimum delays BGDLmin _m , instead of the above-mentioned M scan response delays RTT _m , M packet loss rates PLR _m , M scan maximum delays DLmax _{m ,} and M scan minimum delays DLmin _m .

[Selection of the scan response delay RTT _m _known that is most similar to the scan response delay RTT _m _unknown]
The extraction means 15 detects the scan response delays RTT ₁ to RTT _M as scan response delays RTT ₁ _unknown to RTT _M _unknown from the correspondence table TBL1 (or correspondence table TBL2) stored in the host information database 13. In this case, each of the scan response delays RTT ₁ to RTT _M is composed of one scan response delay or an average value of multiple scan response delays within one IP address Add _m .

In addition, the extraction means 15 detects scan response delay {RTT _m' _known} _1 (m' = 1' to M'), scan response delay {RTT _m' _known} _2 (m' = 1' to M'), and scan response delay {RTT _m' _known} _3 (m' = 1' to M') from the correspondence table TBL3 stored in the host information database 13.

In addition, in FIG. 5, the scan response delays {RTT _m' _known} _1 (m'=1' to M'), {RTT _m' _known} _2 (m'=1' to M'), and {RTT _m' _known} _3 (m'=1' to M') are shown, but the scan response delays {RTT _m' _known} _1 (m'=1' to M'), {RTT _m' _known} _2 (m'=1' to M'), and {RTT _m' _known} _3 (m'=1' to M') are respectively the aggregated scan response delays {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3), so that extraction means 15 updates scan response delay {RTT _m' _known}_1 (m'=1' to M'), scan response delay {RTT _m' _known}_2 (m'=1' to M') and scan response delay {RTT _m' _known}_z (m'=1' to M', z=1 to 3) instead of scan response delay {RTT _m' _known}_1 (m'=1' to M'), scan response delay {RTT _m' _known}_2 (m'=1' to M') and scan response delay {RTT _m' _known_aggregation}_3 (m'=1' to M') may also be detected.

Then, extraction means 15 outputs scan response delays RTT ₁ _unknown to RTT _M _unknown, scan response delay {RTT _m' _known} _1 (m'=1' to M'), scan response delay {RTT _m' _known} _2 (m'=1' to M') and scan response delay {RTT _m' _known} _3 (m'=1' to M') to calculation means 16.

Calculation means 16 receives scan response delays RTT ₁ _unknown to RTT _M _unknown, scan response delay {RTT _m' _known} _1 (m'=1' to M'), scan response delay {RTT _m' _known} _2 (m'=1' to M') and scan response delay {RTT _m' _known} _3 (m'=1' to M') from extraction means 15 .

Calculation means 16 acquires mode m (Mo), which is the most likely value of the scan response delay, based on scan response delay RTT _m . When there are two most likely values of the scan response delay, calculation means 16 calculates mode m (Mo), which is the most likely value of the scan response delay, by the above-mentioned formula (9). When there are three or more most likely values of the scan response delay, calculation means 16 detects the minimum and maximum values from the three or more most likely scan response delays, and calculates mode m (Mo), which is the most likely value of the scan response delay, by the above-mentioned formula (9) based on the detected minimum and maximum values.

In this case, in formula (9), L indicates the lower boundary of the probability density function region (=PDF (Probability Density Function) region) including two values of the most likely scan response delay when the result of the network scan is unknown or when the result of the network scan is known. Furthermore, r indicates the number of samples (=the number of scan response delays) of the most likely PDF region when the result of the network scan is unknown or when the result of the network scan is known. Furthermore, r ₁ indicates the number of samples (=the number of scan response delays) of the previous PDF region in the direction in which the scan response delay RTT _m is smaller than the most likely PDF region when the result of the network scan is unknown or when the result of the network scan is known. Furthermore, r ₂ indicates the number of samples (=the number of scan response delays) of the next PDF region in the direction in which the scan response delay RTT _m is larger than the most likely PDF region when the result of the network scan is unknown or when the result of the network scan is known. Furthermore, h indicates the interval between each PDF region (i.e., the width of each PDF region).

Figures 50 to 52 show specific distributions of scan response delays. Figure 53 shows the correspondence between PDF regions and the number of samples generated. Figure 54 shows a specific distribution of the probability density function of scan response delays.

When the number of samples for each specific value in the specific distribution of scan response delay shown in Figure 50 is counted and the number of samples for each specific value is displayed, the result is as shown in Figure 51. As a result, Figure 51 shows that scan response delay is most likely to occur at probability density values of 0.41 and 0.6.

As shown in FIG. 52, the number of samples generated in the PDF region of 0.0 to 0.2, the PDF region of 0.21 to 0.4, the PDF region of 0.41 to 0.6, the PDF region of 0.61 to 0.8, and the PFD region of 0.81 to 1.0 are "5", "5", "14", "4", and "2", respectively.

The correspondence between the PDF regions and the number of samples generated in each region is as shown in Figure 53. The number of samples generated in the PDF region of 0.41 to 0.6 (=14) is r, the interval of 0.41 to 0.6 (=0.2) is h, 0.41 is L, the number of samples generated in the PDF region of 0.21 to 0.4 (=5) is _r1 , and the number of samples generated in the PDF region of 0.61 to 0.8 (=4) is _r2 . Therefore, the mode m (Mo) can be calculated by substituting r (=14), _r1 (=5), _r2 (=4), h (=0.2), and L (=0.41) into formula (9).

As shown in FIG. 54, r is the number of samples of the scan response delay in the PDF region REG _PDF surrounded by the two values (=0.41 and 0.6) that are most likely to cause the scan response delay and the curve showing the probability density function. Furthermore, L is the lower value (=0.41) of the two values (=0.41 and 0.6) that are most likely to cause the scan response delay. Furthermore, h is the interval between the two values (=0.41 and 0.6). r ₁ is the number of samples in a region of width h adjacent to the PDF region REG _PDF on the side where the scan response delay is smaller than the _PDF region REG _PDF , and r ₂ is the number of samples in a region of width h adjacent to the PDF region REG PDF on the side where the scan response delay is larger than the PDF region REG _PDF .

FIG. 55 is a diagram showing another specific distribution of scan response delay. Referring to FIG. 55, there are three most likely scan response delay values. In this case, the interval h is adjusted to the width of the PDF region including the three most likely scan response delay values. Then, the minimum value (=0.35) and the maximum value (=0.6) are detected from the three most likely scan response delay values. After that, the minimum value (=0.35) and the maximum value (=0.6) are used as the two most likely scan response delay values to determine r, r ₁ , r ₂ , L and h by the method described in FIG. 50 to FIG. 54, and the determined r, r ₁ , r ₂ , L and h are substituted into formula (9) to calculate the mode m (Mo).

When there are four or more most likely scan response delay values, the mode m (Mo) is calculated in the same manner as when there are three most likely scan response delay values.

Hereinafter, the mode m(Mo) calculated using equation (9) based on the scan response delays RTT ₁ _unknown to RTT _M _unknown will be denoted as “m(Mo)_unknown”, the mode m(Mo) calculated using equation (9) based on the scan response delays {RTT _m′ _known} _1 (m′=1′ to M′) will be denoted as “m(Mo)_known_1”, the mode m(Mo) calculated using equation (9) based on the scan response delays {RTT _m′ _known} _2 (m′=1′ to M′) will be denoted as “m(Mo)_known_2”, and the mode m(Mo) calculated using equation (9) based on the scan response delays {RTT _m′ _known}_3 (m'=1' to M'), the mode m(Mo) calculated by equation (9) is represented as "m(Mo)_known_3".

The calculation means 16 calculates the modes m(Mo)_unknown, m(Mo)_known_1, m(Mo)_known_2, and m(Mo)_known_3 by the above-mentioned method, and then calculates the mean absolute deviation W′ _unknown of the scan response delays RTT ₁ _unknown to RTT _M _unknown by the following equation.

C(x _{i — unknown} ) in equation (21A) is expressed by equation (21B), and scan response delays RTT ₁ _unknown to RTT _M _unknown are substituted for x _{i _unknown} .

After calculating the mean absolute deviation _W'unknown , the calculation means 16 calculates the mean absolute deviation _{W'known_z} of the scan response delay {RTT _m' _known}_z (m'=1' to M') by the following equation.

In formula (22), z is an argument indicating the known estimated results of the network scan, and is an integer satisfying 1≦z≦Z. Furthermore, Z is the total number of known estimated results of the network scan. In this embodiment of the invention, the known estimated results of the network scan are "normal", "poor communication quality", and "network congestion", so in formula (22), z = 1, 2, 3.

The scan response delays corresponding to the known estimated network scan results of "normal", "poor communication quality" and "network congestion" may consist of one type of data or multiple types of data.

The calculation means 16 substitutes the mode m(Mo)_known_1 into m(Mo)_known_z in equation (22B) and substitutes the scan response delays {RTT _1′ _known}_1 to {RTT _M′ _known}_1 into x _{i _known_z} in equation (22B) to calculate the mean absolute deviation W′ _{known_1} according to equation (22A).

In addition, the calculation means 16 substitutes the mode m(Mo)_known_2 into m(Mo)_known_z in equation (22B) and substitutes the scan response delays {RTT _1' _known}_2 to {RTT _M' _known}_2 into x _{i _known_z} in equation (22B) to calculate the mean absolute deviation W' _{known_2} by equation (22A).

Furthermore, the calculation means 16 substitutes the mode m(Mo)_known_3 into m(Mo)_known_z in equation (22B) and substitutes the scan response delays {RTT _1′ _known}_3 to {RTT _M′ _known}_3 into x _{i _known_z} in equation (22B) to calculate the mean absolute deviation W′ _{known_3} according to equation (22A).

That is, the calculation means 16 calculates the mean absolute deviations W' known_1, W' known_2, and _W' known_3 based on the modes m(Mo)_known_1, m(Mo)_known_2, and m(Mo)_known_3 and the scan response delays {RTT _{1 '} _known}_1 to {RTT _M' _known}_1, {RTT _1' _known}_2 to {RTT _{M' _known} }_2, and {RTT 1' _{_known} }_3 to {RTT M' _{_known} }_3 using equation (22).

Thereafter, the calculation means 16 substitutes the mean absolute deviations _W'unknown , _{W'known_1} , _{W'known_2} , and _{W'known_3} into the following equation to calculate the differences S _S1 , S _S2 , and S _S3 in the mean absolute deviations.

In this case, the calculation means 16 substitutes the mean absolute deviation W' _{known_1} into W' _{known_z} in equation (23) and substitutes the mean absolute deviation W' _unknown into W' _unknown in equation (23) to calculate the difference S _S1 in the mean absolute deviations.

Furthermore, the calculation means 16 substitutes the mean absolute deviation W' _{known -- 2} into W' _{known -- z} in equation (23) and substitutes the mean absolute deviation W' _unknown into W' _unknown in equation (23) to calculate the difference S _S2 in the mean absolute deviations.

Furthermore, the calculation means 16 substitutes the mean absolute deviation _{W'known_3} into _{W'known_z} in equation (23) and substitutes the mean absolute deviation _W'unknown into _W'unknown in equation (23) to calculate the difference S _S3 in the mean absolute deviations.

After calculating the differences S _S1 , S _S2 , and S _S3 in the mean absolute deviations, the calculation means 16 substitutes the calculated differences S _S1 , S _S2 , and S _S3 in the mean absolute deviations into the following equation to calculate the similarity P _Sz .

In this case, the calculation means 16 calculates the similarity P _S1 by substituting the difference S _S1 in the mean absolute deviations into S _Sz in formula (24), calculates the similarity P S2 by substituting the difference S _S2 in the mean absolute deviations into S _Sz in formula (24), and calculates the similarity P _S3 by substituting the difference S _S3 in the mean absolute deviations into S _Sz in formula (24 ₎ .

After calculating the similarities P _S1 , P _S2 , and P _S3 , the calculation means 16 outputs the calculated similarities P _S1 , P _S2 , and P _S3 to the extraction means 15 .

When extracting means 15 receives similarities P _S1 , P _S2 , and P _S3 from calculating means 16, it detects the highest similarity P _{S_highest} among similarities P _S1 , P _S2 , and P _S3 . Then, extracting means 15 refers to correspondence table TBL3 stored in host information database 13 and detects the _scan response delay {RTT _m' _known}_similarity (m'=1' to M') (= any of scan response delay {RTT m' _known}_1 (m'=1' to M'), scan response delay {RTT _{m '} _known}_2 (m'=1' to M'), and scan response delay {RTT _m' _known}_3 (m'=1' to M')) when the highest similarity P S_highest is obtained. In addition, the extraction means 15 detects a communication characteristic {CMbest _m' _known} _similarity (m' = 1' to M') (= any one of communication characteristic {CMbest _m' _known} _1 (m' = 1' to M'), communication characteristic {CMbest _m' _known} _2 (m' = 1' to M'), and communication characteristic {CMbest _m' _known} _3 (m' = 1' to M')) used to estimate a known estimation result of a network scan together with a scan response delay {RTT _m' _known} _similarity (m' = 1' to M').

Then, the extraction means 15 generates standard sample data D_similarity consisting of the scan response delay {RTT _m' _known}_similarity (m' = 1' to M') and the communication characteristic {CMbest _m' _known}_similarity (m' = 1' to M'), and outputs the generated standard sample data D_similarity to the estimation means 17.

As described above, the scan response delays {RTT _m' _known}_1 (m'=1' to M'), {RTT _m' _known}_2 (m'=1' to M'), and {RTT _m' _known}_3 (m'=1' to M') are updated by the aggregated scan response delay {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3), respectively. Therefore, extraction means 15 extracts the aggregated scan response delay {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3) and the aggregated scan response delay {RTT _m' The estimation means 17 may generate standard sample data D_similarity consisting of communication characteristics {CMbest _m' _known}_z (m' = 1' to M', z = 1 to 3) corresponding to the communication characteristics {CMbest m' _known}_z (m' = 1' to M', z = 1 to 3), and output the generated standard sample data D_similarity to the estimation means 17.

In addition, the extraction means 15 refers to correspondence table TBL1 (or correspondence table TBL2) stored in the host information database 13 to detect scan response delays _{RTT1_unknown} to _{RTTM_unknown} (=scan response delays _RTTm (m=1 to M)), generates sample data D_unknown consisting of the detected scan response delays _{RTT1_unknown} to RTTM_unknown (=scan response delays _RTTm ( _m =1 to M)) and communication characteristics CMbest_unknown_m (m=1 to M), and outputs the generated sample data D_unknown to the estimation means 17.

The estimation means 17 receives the standard sample data D_similarity and the sample data D_unknown from the extraction means 15.

The estimation means 17 then updates the estimation algorithm ALG to the optimal algorithm using the standard sample data D_similarity in the manner described below.

Figure 56 is a diagram showing data when training the estimation algorithm ALG. With reference to Figure 56, when training the estimation algorithm ALG using the standard sample data D_similarity, the estimation means 17 divides the standard sample data D_similarity into training data and test data. In this case, the estimation means 17 divides the standard sample data D_similarity into training data and test data so that the proportion of training data is 80% and the proportion of test data is 20%, for example. Note that when dividing the standard sample data D_similarity into training data and test data, the proportion of training data may be other than 80%, and the proportion of test data may be other than 20%.

Then, the estimation means 17 divides the training data into a training set and a validation set. In this case, the estimation means 17 divides the training data into a training set and a validation set so that the proportion of the training set is 50% and the proportion of the validation set is 50%, for example. Note that the proportions of the training set and the validation set may be other than 50%.

The estimation means 17 divides the training data into a training set and a validation set, and then divides the training set into U subsets d ₁ to d _U. Each of the subsets d ₁ to d _U is made up of data of size n. Here, n is, for example, 8. The estimation means 17 also divides the validation set into U' subsets V _1' to V _U' . Then, if the data size of the training set is D _S , then U=INT(D _S /n).

Fig. 57 is a conceptual diagram of sample data when training the estimation algorithm ALG. Referring to Fig. 57, when training the estimation algorithm ALG, the estimation means 17 creates initial sample data for modeling. More specifically, estimation means 17 reads out scan response delays {RTT _1' _known}_1 to {RTT _M' _known}_1 and communication characteristics {CMbest _1' _known}_1 to {CMbest _M' _known}_1 from correspondence table TBL3 shown in FIG. 3, and generates standard sample data D_standard_1 from the read scan response delays {RTT _1' _known}_1 to {RTT _M' _known}_1 and communication characteristics {CMbest _1' _known}_1 to {CMbest _M' _known}_1.

In addition, estimation means 17 reads out scan response delays {RTT _1' _known}_2 to {RTT _M' _known}_2 and communication characteristics {CMbest _1' _known}_2 to {CMbest _M' _known}_2 from correspondence table TBL3, and generates standard sample data D_standard_2 from the read scan response delays {RTT _1' _known}_2 to {RTT _M' _known}_2 and communication characteristics {CMbest _1' _known}_2 to {CMbest _M' _known}_2.

Furthermore, estimation means 17 reads out scan response delays {RTT _1' _known}_3 to {RTT _M' _known}_3 and communication characteristics {CMbest _1' _known}_3 to {CMbest _M' _known}_3 from correspondence table TBL3, and generates standard sample data D_standard_3 from the read scan response delays {RTT _1' _known}_3 to {RTT _M' _known}_3 and communication characteristics {CMbest _1' _known}_3 to {CMbest _M' _known}_3.

Then, the estimation means 17 generates the initial sample data D_sample_initial as follows (see FIG. 57(a)). In this case, the estimation means 17 selects one subset d 1 _{_initial} from the multiple subsets d _{1 _1} to d _{U _1} of the standard sample data D_standard_1, selects one subset d 2 _initial from the multiple subsets d _{1 _2} to d _{U _2} of the standard sample data _{D_standard_2} , and selects one subset d 3 _{_initial} from the multiple subsets d _{1 _3} to d _{U _3} of the standard sample data D_standard_3. Then, "1", "2", and "3" in the standard sample data D_standard_1, D_standard_2, and D_standard_3 respectively represent the identification code ID_code=1 indicating that the result of the network scan is "normal", the identification code ID_code=2 indicating that the result of the network scan is "insufficient communication quality", and the identification code ID_code=3 indicating that the result of the network scan is "network congestion", and therefore the estimation means 17 selects the subsets _{d1_initial} , _{d2_initial} , and _{d3_initial} , and combines the subsets _{d1_initial} , d2_initial, and _{d3_initial} via the identification codes ID_code=1, ₂ , and 3 to generate initial sample data D_sample_initial.

In addition, "1" of the multiple subsets d _{1_1} to d _{U_1} corresponds to "1" of the standard sample data D_standard_1, "2" of the multiple subsets d _{1_2} to d _{U_2} corresponds to "2" of the standard sample data D_standard_2, and "3" of the multiple subsets d _{1_3} to d _{U_3} corresponds to "3" of the standard sample data D_standard_3. In addition, one subset d _{1_initial} , d _{2_initial} , d 3_initial may be selected randomly from the multiple subsets d _{1_1} to d _{U_1} , the multiple subsets d _{1_2 to} d _{U_2} , and the multiple subsets d _{1_3} to d _{U_3} , respectively, or may be selected according to any selection method.

56, subsets d ₁ to d _U are generated from the standard sample data D_similarity. The standard sample data D_similarity is the sample data that has the highest similarity to the sample data D_unknown among the standard sample data D_standard_1 to D_standard_3.

As a result, the subsets d ₁ to d _U are generated from the standard sample data D_standard_1 by the method described in Fig. 56 when the standard sample data D_standard_1 has the highest similarity to the sample data D_unknown. Also, the subsets d ₁ to d _U are generated from the standard sample data D_standard_2 by the method described in Fig. 56 when the standard sample data D_standard_2 has the highest similarity to the sample data D_unknown. Furthermore, the subsets d ₁ to d _U are generated from the standard sample data D_standard_3 by the method described in Fig. 56 when the standard sample data D_standard_3 has the highest similarity to the sample data D_unknown.

When the subsets d ₁ to d _U are generated from the standard sample data D_standard_1, the standard sample data D_standard_1 in the initial sample data D_sample_initial for modeling is replaced by the subset d _u to generate tentative sample data D_sample_tentative (D1) for modeling (see FIG. 57(b)). Note that "D1" in D_sample_tentative (D1) indicates that the standard sample data D_standard_1 in the initial sample data D_sample_initial has been replaced by the subset d _u .

Furthermore, when the subsets d ₁ to d _U are generated from the standard sample data D_standard_2, the standard sample data D_standard_2 in the initial sample data D_sample_initial for modeling is replaced by the subset d _u to generate tentative sample data D_sample_tentative (D2) for modeling (see FIG. 57(c)). Note that "D2" in D_sample_tentative (D2) indicates that the standard sample data D_standard_2 in the initial sample data D_sample_initial has been replaced by the subset d _u .

Furthermore, when the subsets d ₁ to d _U are generated from the standard sample data D_standard_3, the standard sample data D_standard_3 in the initial sample data D_sample_initial for modeling is replaced by the subset d _u to generate tentative sample data D_sample_tentative (D3) for modeling (see FIG. 57(d)). Note that "D3" in D_sample_tentative (D3) indicates that the standard sample data D_standard_3 in the initial sample data D_sample_initial has been replaced by the subset d _u .

Fig. 58 is a conceptual diagram showing an update of tentative sample data for modeling. In Fig. 58, a method of updating tentative sample data for modeling D_sample_tentative (D1) is described using an example in which subsets d ₁ to d _U are generated from standard sample data D_standard_1.

58, the estimation means 17 first creates initial sample data D_sample_initial for modeling by the above-mentioned method. After that, the estimation means 17 replaces the standard sample data D_standard_1 of the sample data D_sample_initial by the subset d ₁ to update the initial sample data D_sample_initial to tentative sample data D_sample_tentative_1 for modeling. Then, by updating the initial sample data D_sample_initial to the tentative sample data D_sample_tentative_1, the initial sample data D_sample_initial becomes the "old training sample" and the tentative sample data D_sample_tentative_1 becomes the "new training sample".

Then, if the estimation means 17 determines that the evaluation index when estimating the results of the network scan using the tentative sample data D_sample_tentative_1 (= "new training sample") is better than the evaluation index when estimating the results of the network scan using the initial sample data D_sample_initial (= "old training sample"), the tentative sample data D_sample_tentative_1 becomes the "old training sample."

Next, the estimation means 17 updates the tentative sample data D_sample_tentative_1 to tentative sample data D_sample_tentative_2 by replacing the subset d ₁ of the tentative sample data D_sample_tentative_1 with the subset d _2. Then, if the evaluation index when the result of the network scan is estimated using the tentative sample data D_sample_tentative_2 (= "new training sample") is better than the evaluation index when the result of the network scan is estimated using the "old training sample", the tentative sample data D_sample_tentative_2 becomes the "old training sample". In addition, when comparing the evaluation index between the tentative sample data D_sample_tentative_2 (= "new training sample") and the "old training sample", the "old training sample" is made up of the tentative sample data D_sample_tentative_1 if, in the previous comparison of evaluation indices, the evaluation index when the result of the network scan was estimated using the tentative sample data D_sample_tentative_1 was better than the evaluation index when the result of the network scan was estimated using the initial sample data D_sample_initial, and is made up of the initial sample data D_sample_initial if the evaluation index when the result of the network scan was estimated using the initial sample data D_sample_initial was better than the evaluation index when the result of the network scan was estimated using the tentative sample data D_sample_tentative_1. In other words, the sample data with the better evaluation index will become the "old training sample" to be compared in the next comparison of evaluation indexes.

Furthermore, the estimation means 17 continues by replacing the subset d ₂ of the tentative sample data D_sample_tentative_2 with the subset d ₃ , thereby updating the tentative sample data D_sample_tentative_2 to the tentative sample data D_sample_tentative_3. Then, if the evaluation index when the result of the network scan is estimated using the tentative sample data D_sample_tentative_3 is better than the evaluation index when the result of the network scan is estimated using the "old training sample", the tentative sample data D_sample_tentative_3 becomes the "old training sample".

The estimation means 17 repeats the process of updating the provisional sample data D_sample_tentative_u to the provisional sample data D_sample_tentative_u+1 in a similar manner, and finally updates it to the provisional sample data D_sample_tentative_U.

In this way, the estimation means 17 successively replaces the standard sample data D_standard — 1 of the initial sample data D_sample_initial with U subsets d ₁ to d _U , thereby successively updating the initial sample data D_sample_initial to tentative sample data D_sample_tentative — 1 to D_sample_tentative_U.

Then, when the standard sample data D_standard_1 has the highest similarity to the sample data D_unknown, the initial sample data D_sample_initial and the tentative sample data D_sample_tentative_1 to D_sample_tentative_U are used to train the estimation algorithm ALG.

When the standard sample data D_standard_2 has the highest similarity to the sample data D_unknown, the tentative sample data D_sample_tentative (D2) shown in (c) of Figure 57 constitutes the tentative sample data D_sample_tentative_1, and the estimation means 17 sequentially updates the initial sample data D_sample_initial to tentative sample data D_sample_tentative_1 to D_sample_tentative_U using the method described in Figure 58.

Furthermore, when the standard sample data D_standard_3 has the highest similarity to the sample data D_unknown, the tentative sample data D_sample_tentative (D3) shown in (d) of Figure 57 constitutes the tentative sample data D_sample_tentative_1, and the estimation means 17 sequentially updates the initial sample data D_sample_initial to tentative sample data D_sample_tentative_1 to D_sample_tentative_U by the method described in Figure 58.

As described above, the initial sample data D_sample_initial is composed of the standard sample data D_standard_1 to D_standard_3 combined, and the tentative sample data D_sample_tentative_1 to D_sample_tentative_U are composed of any of the standard sample data D_standard_1 to D_standard_3 in the initial sample data D_sample_initial replaced by a subset d ₁ to d _U , depending on which of the standard sample data D_standard_1 to D_standard_3 the standard sample data D_similarity is composed of. The subsets d ₁ to d _U are each composed of any one of the standard sample data D_standard_ 1 to D_standard_ 3 that make up the standard sample data D_similarity.

Therefore, the initial sample data D_sample_initial and the tentative sample data D_sample_tentative_1 to D_sample_tentative_U are basically composed of a combination of parts of the standard sample data D_standard_1 to D_standard_3.

The initial sample data D_sample_initial and the tentative sample data D_sample_tentative_1 to D_sample_tentative_U are composed of the standard sample data D_standard_1 to D_standard_3 combined for the following reasons.

As described later, the estimation algorithm ALG classifies a group _CLS1 in which the result of the network scan is “normal”, a group _CLS2 in which the result of the network scan is “inadequate communication quality”, and a group CLS3 in which the result of the network scan is “network congestion”. ₃ , the initial sample data D_sample_initial and the tentative sample data D_sample_tentative_1 to D_sample_tentative_U, which are data used when training the estimation algorithm ALG, are composed of standard sample data D_standard_1 acquired in a communications environment where the network scan result is "normal", standard sample data D_standard_2 acquired in a communications environment where the network scan result is "inadequate communication quality", and standard sample data D_standard_3 acquired in a communications environment where the network scan result is "network congestion", so that the network scan result can be accurately estimated as any of "normal", "inadequate communication quality", and "network congestion".

In addition, in Figures 56 to 58, it has been explained that the standard sample data D_standard_1, D_standard_2, and D_standard_3 are generated using the scan response delays {RTT _{m '} _known}_1 (m' = 1' to M'), {RTT _m' _known}_2 (m' = 1' to M'), and {RTT _m' _known}_3 (m' = 1' to M') stored in correspondence table TBL3, respectively. However, the standard sample data D_standard_1 may be generated using the scan response delay {RTT _m' _known_aggregation}_1 (m' = 1' to M') instead of the scan response delay {RTT _m' The standard sample data D_standard_2 may be generated using a scan response delay {RTT _m' _known_aggregation}_2 (m'=1' to M') instead of the scan response delay {RTT _m' _known}_2 (m'=1' to M'), and the standard sample data D_standard_3 may be generated using a scan response delay {RTT _{m' _known_aggregation}_3 (m'=1' to M') instead of the scan response delay {RTT m'} _known}_3 (m'=1' to M').

As a result, the aggregated scan response delays {RTT _m' _known_aggregation}_z (m' = 1' to M', z = 1 to 3) are used to generate the training set subsets d ₁ to d _U , the validation set subsets V _1' to V _U' , and the test data (see FIG. 56 ), and the estimation algorithm ALG is trained using the aggregated scan response delays {RTT _m' _known_aggregation}_z (m' = 1' to M', z = 1 to 3), and the training results are evaluated.

Figure 59 is a conceptual diagram of training and evaluation of the estimation algorithm ALG. Referring to Figure 59, the estimation means 17 trains the estimation algorithm ALG using the initial sample data D_sample_initial. In this case, the estimation means 17 trains the estimation algorithm ALG by repeatedly estimating the results of the network scan by the estimation algorithm ALG using the initial sample data D_sample_initial.

Then, the estimation means 17 estimates the results of the network scan using the validation set. After that, the estimation means 17 evaluates the estimated results of the network scan using the evaluation index EVD.

Subsequently, the estimation means 17 performs the following steps for all of the tentative sample data D_sample_tentative_1 to D_sample_tentative_U: (I) training the estimation algorithm ALG using the tentative sample data D_sample_tentative_u; (II) estimating the results of the network scan using a validation set (= the entire subsets V1 _' to VU _' ); and (III) evaluating the estimated results of the network scan using an evaluation index nEVD_u. In this case, the estimation means 17 determines whether the evaluation index nEVD_u is greater than the evaluation index EVD each time it executes (I) to (III), and when the evaluation index nEVD_u is greater than the evaluation index EVD, it sets the evaluation index nEVD_u to the evaluation index EVD and updates the evaluation index EVD with the evaluation index nEVD_u, and when the evaluation index nEVD_u is not greater than the evaluation index EVD, it does not update the evaluation index EVD (i.e., it maintains the evaluation index EVD). Then, the estimation means 17 maintains the variables of the estimation algorithm ALG when the evaluation index EVD is the best when the result of the network scan is estimated using the initial sample data D_sample_initial, and replaces the variables of the estimation algorithm ALG with the provisional sample data (any of the provisional sample data D_sample_tentative_1 to D_sample_tentative_U) when the largest evaluation index nEVD_u is obtained when the evaluation index nEVD_u is the best when the result of the network scan is estimated using the provisional sample data (any of the provisional sample data D_sample_tentative_1 to D_sample_tentative_U).

Then, the estimation means 17 estimates the results of the network scan using the test data, and evaluates whether to approve the training model (estimation algorithm ALG after training) and the training dataset based on the evaluation index tEVD. More specifically, when the evaluation index tEVD is equal to or greater than the evaluation index EVD (finally updated evaluation index EVD), the estimation means 17 approves the training model (estimation algorithm ALG after training) and the training dataset, and when the evaluation index tEVD is not equal to or greater than the evaluation index EVD (finally updated evaluation index EVD), the estimation means 17 does not approve the training model (estimation algorithm ALG after training) and the training dataset. Then, when the estimation means 17 does not approve the training model (estimation algorithm ALG after training) and the training dataset, (I) to (IV) shown in FIG. 59 are repeatedly executed using the estimation algorithm ALG with the variables finally replaced.

The approved training data set becomes the initial sample data D_sample_initial when the next training and evaluation are performed. The evaluation index EVD is composed of the F1 score _F1 or the Matthews correlation coefficient MCC described later, the evaluation index nEVD_u is composed of the F1 score _nF1 or the Matthews correlation coefficient nMCC described later, and the evaluation index tEVD is composed of the F1 score _tF1 or the Matthews correlation coefficient tMCC described later.

In this way, the estimation means 17 trains the estimation algorithm ALG using the initial sample data D_sample_initial and the tentative sample data D_sample_tentative_1 to D_sample_tentative_U, and evaluates the estimation results when the network scan results are estimated by the trained estimation algorithm ALG using a validation set (= consisting of all of the subsets V1 _' to VU _' ) using the evaluation indexes EVD and nEVD, and finally, uses the test data to evaluate whether the trained estimation algorithm ALG and the training data set are to be approved using the evaluation index tEVD.

Figure 60 is a diagram showing a specific example of the initial sample data D_sample_initial for modeling shown in Figure 58. Figure 61 is a diagram showing a specific example of the training set shown in Figure 56. Figure 62 is a diagram showing a specific example of the validation set shown in Figure 56. Figure 63 is a diagram showing a specific example of the test data shown in Figure 56.

Note that in Figures 60 to 63, a case is taken as an example in which the number of terminal devices to be subjected to the network scan is 8 (i.e., M' = 8) and the number of samples of the standard sample data D_similarity is 20 for one IP address Add _m' , and specific examples of the initial sample data D_sample_initial, a training set, a validation set, and test data are shown, respectively.

60, the standard sample data D_standard_1 is composed of scan response delays RTT _m' and communication characteristics CMbest _m' when a network scan is executed in a communication environment where the result of the network scan is "normal." Therefore, in the standard sample data D_standard_1, the eight specific values of the scan response delays RTT _m' and the eight specific values of the communication characteristics CMbest _m' are a subset of the standard sample data D_standard_1 when M'=8 in the scan response delays {RTT _1' _known}_1 to {RTT _M' _known}_1 stored in the correspondence table TBL3 shown in FIG.

Moreover, the standard sample data D_standard_2 is composed of scan response delays RTT _m' and communication characteristics CMbest _m' when a network scan is performed in a communication environment where the result of the network scan is "insufficient communication quality." Therefore, in the standard sample data D_standard_2, the eight specific values of the scan response delays RTT _m' and the eight specific values of the communication characteristics CMbest _m' are a subset of the standard sample data D_standard_2 when M'=8 in the scan response delays {RTT _1' _known}_2 to {RTT _M' _known}_2 stored in the correspondence table TBL3.

Furthermore, the standard sample data D_standard_3 is composed of scan response delays RTT _m' and communication characteristics CMbest _m' when a network scan is performed in a communication environment where the result of the network scan is "network congestion." Therefore, in the standard sample data D_standard_3, the eight specific values of the scan response delays RTT _m' and the eight specific values of the communication characteristics CMbest _m' are a subset of the standard sample data D_standard_3 when M'=8 in the scan response delays {RTT _1' _known}_3 to {RTT _M' _known}_3 stored in the correspondence table TBL3.

As a result, the initial sample data D_sample_initial for modeling is composed of the standard sample data D_standard_1 to D_standard_3 combined with the identification code D_code=1 assigned to the network scan result "normal," the identification code D_code=2 assigned to the network scan result "poor communication quality," and the identification code D_code=3 assigned to the network scan result "network congestion."

61, the training set consists of eight subsets d ₁ to d _8. Since M′=8, each of the subsets d ₁ to d ₈ consists of eight scan response delays RTT _1′ to RTT _8′ and eight communication characteristics CMbest _1′ to CMbest _8′ . Therefore, in each of the subsets d ₁ to d ₈ , the vertical direction indicates eight scan response delays RTT _1′ to RTT _8′ corresponding to eight terminal devices and eight communication characteristics CMbest _1′ to CMbest _8′ .

The number of subsets d ₁ to d ₈ is eight because the number of samples in the standard sample data D_similarity is 20, which is 20×0.8×0.5=8.

5. Furthermore, when the standard sample data D_similarity is made up of the standard sample data D_standard_1, each of the subsets d ₁ to d ₈ is made up of the scan response delays {RTT _{1 ′} _known}_1 to {RTT _M′ _known}_1 and the communication characteristics {CMbest _{1 ′} _known}_1 to {CMbest _M′ _known}_1 stored in the correspondence table TBL3 shown in FIG. 5. When the standard sample data D_similarity is made up of the standard sample data D_standard_2, each of the subsets d 1 to d 8 is made up of the scan response delays {RTT _{1 ′} _known}_2 to {RTT _M′ _known}_2 and the communication characteristics {CMbest _{1 ′} _known}_2 to {CMbest _M′ _known}_2 and the standard sample data D_similarity is standard sample data D_standard_3, the correspondence table TBL3 shown in FIG. 5 contains scan response delays {RTT _1' _known}_3 to {RTT _M' _known}_3 and communication characteristics {CMbest _1' _known}_3 to {CMbest _M' _known}_3.

Furthermore, when the subsets d ₁ to d ₈ each consist of scan response delays {RTT _1′ _known} — 1 to {RTT _8′ _known} — 1 and communication characteristics {CMbest _1′ _known} — 1 to {CMbest _8′ _known} — 1, the standard sample data D_standard — 1 of the initial sample data D_sample_initial shown in FIG. 60 is sequentially replaced with the subsets d ₁ to d ₈ , thereby sequentially generating tentative sample data D_sample_tentative — 1 to D_sample_tentative_U (U=8) shown in FIG. 58.

Furthermore, when the subsets d ₁ to d ₈ each consist of scan response delays {RTT _1′ _known} — 2 to {RTT _8′ _known} — 2 and communication characteristics {CMbest _1′ _known} — 22 to {CMbest _8′ _known} — 2, the standard sample data D_standard_2 of the initial sample data D_sample_initial shown in FIG. 60 is replaced sequentially with the subsets d ₁ to d ₈ , thereby sequentially generating tentative sample data D_sample_tentative_1 to D_sample_tentative_U (U=8) shown in FIG. 58.

Furthermore, when the subsets d ₁ to d ₈ each consist of scan response delays {RTT _1′ _known} — 3 to {RTT _8′ _known} — 3 and communication characteristics {CMbest _1′ _known} — 3 to {CMbest _8′ _known} — 3, the standard sample data D_standard_3 of the initial sample data D_sample_initial shown in FIG. 60 is replaced sequentially with the subsets d ₁ to d ₈ , thereby sequentially generating tentative sample data D_sample_tentative_1 to D_sample_tentative_U (U=8) shown in FIG. 58.

62, the validation set consists of eight subsets V _{1 '} to V _{8 '} . Since M'=8, each of the subsets V _{1 '} to V _{8 '} consists of eight scan response delays RTT _{1 '} to RTT _{8 '} and eight communication characteristics CMbest _{1 '} to CMbest _{8 '} . Therefore, in each of the subsets V _{1 '} to V _{8 '} , the vertical direction indicates eight scan response delays RTT _{1 '} to RTT _{8 '} and eight communication characteristics CMbest _{1 '} to CMbest _{8 '} corresponding to eight terminal devices.

The reason why the number of subsets V _{1 '} to V _{8 '} is eight is the same as the reason why the number of subsets d ₁ to d ₈ is eight.

5. Furthermore, when the standard sample data D_similarity is made up of the standard sample data D_standard_1, each of the subsets V _{1 ′} to V _{8 ′} is made up of a part of the scan response delays {RTT _{1 ′} _known} _1 to {RTT _M′ _known} _1 and a part of the communication characteristics {CMbest _{1 ′} _known} _1 to {CMbest _{M′ _known} _1 stored in the correspondence table TBL3 shown in FIG. 5. When the standard sample data D_similarity is made up of the standard sample data D_standard_2, each of the subsets V 1 ′} to V 8 ′ is made up of a part of the scan response delays {RTT _{1 ′} _known} _2 to {RTT _M′ _known} _2 and a part of the communication characteristics { _CMbest 5. When the standard sample data D_similarity consists of a portion of the scan response delays {RTT _1' _known}_3 to {RTT _M' _known}_3 and a portion of the communication characteristics {CMbest 1' _known}_3 to {CMbest _{M' _known}_3, and when the standard sample data D_similarity consists of the standard sample data D_standard_3, the standard sample data D_similarity consists of a portion of the scan response delays {RTT 1' _known}_3 to {RTT M} ' _known}_3 and a portion of the communication characteristics {CMbest _1' _known}_3 to {CMbest _M' _known}_3.

63, the test data consists of four subsets TD ₁ to TD _4. Since M'=8, each of the subsets TD ₁ to TD ₄ consists of eight scan response delays RTT _1' to RTT _8' and eight communication characteristics CMbest _1' to CMbest _8' . Therefore, in each of the subsets TD ₁ to TD ₄ , the vertical direction indicates eight scan response delays RTT _1' to RTT _8' corresponding to eight terminal devices and eight communication characteristics CMbest _1' to CMbest _8' .

The reason why the number of subsets TD ₁ to TD ₄ is four is that the number of samples of the standard sample data D_similarity is 20 for one IP address Add _m' , and the test data consists of 20% (20×0.2=4) of the number of samples of the standard sample data D_similarity (=20).

5. Furthermore, when the standard sample data D_similarity is made up of the standard sample data D_standard_1, each of the subsets TD ₁ to TD ₄ is made up of a part of the scan response delays {RTT _{1 '} _known}_1 to {RTT _{M'_known} }_1 and a part of the communication characteristics {CMbest _{1 '} _known}_1 to {CMbest _{M'_known} }_1 stored in the correspondence table TBL3 shown in FIG. 5. When the standard sample data D_similarity is made up of the standard sample data D_standard_2, each of the subsets TD 1 to TD 4 is made up of a part of the scan response delays {RTT _{1 '} _known}_2 to {RTT _{M'_known} }_2 and a part of the communication characteristics {CMbest _{1 '} _known}_3 stored in the correspondence table TBL3 shown in FIG. 5. When the standard sample data D_similarity consists of a portion of the scan response delays {RTT _1' _known}_3 to {RTT _M' _known}_3 and a portion of the communication characteristics {CMbest 1' _known}_3 to {CMbest _{M' _known}_3, and when the standard sample data D_similarity consists of the standard sample data D_standard_3, the standard sample data D_similarity consists of a portion of the scan response delays {RTT 1' _known}_3 to {RTT M} ' _known}_3 and a portion of the communication characteristics {CMbest _1' _known}_3 to {CMbest _M' _known}_3.

The estimation means 17 trains an estimation algorithm ALG using the initial sample data D_sample_initial and the tentative sample data D_sample_tentative_1 to D_sample_tentative_U. When training the estimation algorithm ALG using the tentative sample data D_sample_tentative_1 to D_sample_tentative_U, the estimation means 17 sequentially generates tentative sample data D_sample_tentative_1 to D_sample_tentative_U (U=8) including subsets d ₁ to d ₈ , respectively, by the method described in Figures 58 and 61, and trains the estimation algorithm ALG by repeatedly estimating the results of the network scan by the estimation algorithm ALG using the generated tentative sample data D_sample_tentative_1 to D_sample_tentative_U (U=8).

The estimation means 17 trains the estimation algorithm ALG using the initial sample data D_sample_initial, and then estimates the results of the network scan by the estimation algorithm ALG using the validation set, and generates a confusion matrix CFM based on the estimated results of the network scan.

The estimation means 17 also trains the estimation algorithm ALG using the tentative sample data D_sample_tentative_u, and then estimates the results of the network scan by the estimation algorithm ALG using the validation set, and generates a confusion matrix nCFM based on the estimated results of the network scan. The estimation means 17 performs this operation for all of the tentative sample data D_sample_tentative_1 to D_sample_tentative_U.

Figure 64 is a conceptual diagram of a confusion matrix. When estimating the results of a network scan, there are three known estimated results: "normal", "inadequate communication quality", and "network congestion". Therefore, Figure 64 shows a confusion matrix for the three known estimated results: "normal", "inadequate communication quality", and "network congestion".

With reference to Fig. 64, TP indicates the percentage of cases where an assumed event occurred and the occurrence (occurrence of an assumed event) was correctly estimated, TN indicates the percentage of cases where an assumed event did not actually occur and the occurrence (non-occurrence of an assumed event) was correctly estimated, FN indicates the percentage of cases where an assumed event occurred and the occurrence (occurrence of an assumed event) was incorrectly estimated as a different event, and FP indicates the percentage of cases where an assumed event did not actually occur and the occurrence of an assumed event was incorrectly estimated as occurring. Also, in Fig. 64 (a), (b), and (c), "Normal", "Insufficient communication quality", and "Network congestion" in the column direction (vertical direction) represent events that actually occurred, and "Normal", "Insufficient communication quality", and "Network congestion" in the row direction (horizontal direction) represent estimated events. Furthermore, (a) in FIG. 64 shows the confusion matrix when the result of the network scan is "normal," (b) in FIG. 64 shows the confusion matrix when the result of the network scan is "inadequate communication quality," and (c) in FIG. 64 shows the confusion matrix when the result of the network scan is "network congestion."

In FIG. 64(a), the TP at the intersection of the "Normal" row and the "Normal" column indicates the percentage of cases where an assumed "normal" event (the occurrence of an assumed "normal" event) is correctly estimated as a "normal" event, the FN at the intersection of the "Normal" row and the "Poor communication quality" column indicates the percentage of cases where an assumed "normal" event is incorrectly estimated as a different "poor communication quality" event, and the FN at the intersection of the "Normal" row and the "Network congestion" column indicates the percentage of cases where an assumed "normal" event is incorrectly estimated as a different "network congestion" event.

In addition, the FP at the intersection of the row of "inadequate communication quality" and the column of "normal" indicates the percentage of cases where an event of "inadequate communication quality" that is not an assumed "normal" event occurred (i.e., the assumed "normal" event did not actually occur), and the TN at the intersection of the row of "inadequate communication quality" and the column of "inadequate communication quality" indicates the percentage of cases where an event of "inadequate communication quality" that is not an assumed "normal" event occurred, and the TN at the intersection of the row of "inadequate communication quality" and the column of "network congestion ...network congestion" that is not an assumed "normal" event occurred, and the TN at the intersection of the row of "inadequate communication quality" and the column of "network congestion" indicates the percentage of cases where an event of "inadequate communication quality" that is not an assumed "normal" event occurred, and the

Furthermore, the FP at the intersection of the "Network Congestion" row and the "Normal" column indicates the percentage of cases where a "Network Congestion" event that is not an assumed "normal" event occurred (i.e., the assumed "normal" event did not actually occur) was incorrectly estimated to have occurred as an assumed "normal" event, the TN at the intersection of the "Network Congestion" row and the "Inadequate Communication Quality" column indicates the percentage of cases where a "Network Congestion" event that is not an assumed "normal" event occurred was incorrectly estimated to have occurred as an "Inadequate Communication Quality" event (i.e., the assumed "normal" event did not actually occur) was correctly estimated, and the TN at the intersection of the "Network Congestion" row and the "Network Congestion" column indicates the percentage of cases where a "Network Congestion" event that is not an assumed "normal" event occurred was incorrectly estimated to have occurred (i.e., the assumed "normal" event did not actually occur) was correctly estimated.

In FIG. 64 (b), the TN at the intersection of the “Normal” row and the “Normal” column indicates the percentage of cases where a “normal” event occurred that was not the assumed “inadequate communication quality” but was assumed to have occurred (i.e., when the assumed “inadequate communication quality” event did not actually occur, the percentage of cases where the assumption was correctly assumed (i.e., that the assumed “inadequate communication quality” event did not occur)). The FP at the intersection of the “Normal” row and the “inadequate communication quality” column indicates the percentage of cases where a “normal” event occurred that was not the assumed “inadequate communication quality” but was assumed to have occurred (i.e., when the assumed “inadequate communication quality” event did not actually occur, the percentage of cases where the assumption was correctly assumed (i.e., that the assumed “inadequate communication quality” event did not occur)). The TN at the intersection of the "Normal" row and the "Network Congestion" column indicates the percentage of cases where a "Network Congestion" event was incorrectly assumed to have occurred when a "Normal" event occurred that was not the assumed "Inadequate Communication Quality" (i.e., the percentage of cases where a "Inadequate Communication Quality" event did not actually occur but was correctly assumed to have occurred ...)).

FN at the intersection of the row "inadequate communication quality" and the column "normal" indicates the percentage of cases where the occurrence of the assumed "inadequate communication quality" event was incorrectly estimated as a different "normal" event when the assumed "inadequate communication quality" event occurred, TP at the intersection of the row "inadequate communication quality" and the column "inadequate communication quality" indicates the percentage of cases where the occurrence of the assumed "inadequate communication quality" event was correctly estimated as a "inadequate communication quality" event when the assumed "inadequate communication quality" event occurred, and FN at the intersection of the row "inadequate communication quality" and the column "network congestion" indicates the percentage of cases where the occurrence of the assumed "inadequate communication quality" event was incorrectly estimated as a different "network congestion" event when the assumed "inadequate communication quality" event occurred.

Furthermore, the TN at the intersection of the "Network Congestion" row and the "Normal" column indicates the percentage of cases where a "normal" event was estimated to have occurred when a "network congestion" event occurred that was not the assumed "inadequate communication quality" (i.e., when the assumed "inadequate communication quality" event did not actually occur, the percentage of cases where the fact (the assumed "inadequate communication quality" event did not occur) was correctly estimated), the FP at the intersection of the "Network Congestion" row and the "inadequate communication quality" column indicates the percentage of cases where a "network congestion" event occurred that was not the assumed "inadequate communication quality" and the percentage of cases where the incorrectly estimated fact (the assumed "inadequate communication quality" event did not occur) was correctly estimated), the TN at the intersection of the "Network Congestion" row and the "Network Congestion" column indicates the percentage of cases where a "network congestion" event occurred that was not the assumed "inadequate communication quality" and the percentage of cases where the "network congestion" event occurred that was not the assumed "inadequate communication quality" (i.e., when the assumed "inadequate communication quality" event did not actually occur, the percentage of cases where the correct estimation (the assumed "inadequate communication quality" event did not occur) was correctly estimated).

In FIG. 64(c), the TN at the intersection of the "Normal" row and the "Normal" column indicates the percentage of cases where a "normal" event other than the assumed "network congestion" occurred and the assumption that the "normal" event occurred was assumed (i.e., when the assumed "network congestion" event did not actually occur, the assumption that the "network congestion" event did not occur was correctly assumed), the TN at the intersection of the "Normal" row and the "Inadequate communication quality" column indicates the percentage of cases where a "normal" event other than the assumed "network congestion" occurred and the assumption that the "inadequate communication quality" event occurred was assumed (i.e., when the assumed "network congestion" event did not actually occur, the assumption that the "network congestion" event did not occur was correctly assumed), and the FP at the intersection of the "Normal" row and the "Network congestion" column indicates the percentage of cases where a "normal" event other than the assumed "network congestion" occurred and the assumption that the "network congestion" event occurred was incorrectly assumed.

In addition, the TN at the intersection of the row of "inadequate communication quality" and the column of "normal" indicates the percentage of cases where a "normal" event was assumed to have occurred when an "inadequate communication quality" event occurred that was not the assumed "network congestion" (i.e., when the assumed "network congestion" event did not actually occur, the percentage of cases where the event was correctly assumed to have occurred (i.e., the assumed "network congestion" event did not occur)). In addition, the TN at the intersection of the row of "inadequate communication quality" and the column of "inadequate communication quality" indicates the percentage of cases where a "normal" event was assumed to have occurred when an "inadequate communication quality" event occurred that was not the assumed "network congestion" (i.e., when the assumed The FP at the intersection of the "inadequate communication quality" row and the "network congestion" column indicates the percentage of cases where an "inadequate communication quality" event, which is not the assumed "network congestion", was correctly estimated to have occurred (i.e., when the assumed "network congestion" event did not actually occur, the percentage of cases where the FP correctly estimated that the assumed "network congestion" event had occurred) when an "inadequate communication quality" event, which is not the assumed "network congestion" event, occurred (i.e., when the assumed "network congestion" event did not actually occur), the percentage of cases where the FP correctly estimated that the assumed "network congestion" event had occurred.

Furthermore, the FN at the intersection of the "Network Congestion" row and the "Normal" column indicates the percentage of cases where the occurrence of the assumed "Network Congestion" event was mistakenly estimated as a different "Normal" event when the assumed "Network Congestion" event occurred, the FN at the intersection of the "Network Congestion" row and the "Poor Communication Quality" column indicates the percentage of cases where the occurrence of the assumed "Network Congestion" event was mistakenly estimated as a different "Poor Communication Quality" event when the assumed "Network Congestion" event occurred, and the TP at the intersection of the "Network Congestion" row and the "Network Congestion" column indicates the percentage of cases where the occurrence of the assumed "Network Congestion" event was correctly estimated as a "Network Congestion" event when the assumed "Network Congestion" event occurred.

The method of generating a confusion matrix will be explained. Figure 65 is a diagram for explaining the method of generating a confusion matrix. Note that in Figure 65, the method of generating a confusion matrix will be explained using as an example the case of generating a confusion matrix when the result of the network scan shown in Figure 64(a) is "normal".

Referring to FIG. 65, since the result of the network scan is "normal," in the column direction, "normal" is an expected event, while "poor communication quality" and "network congestion" are unexpected events.

The estimation means 17 trains the estimation algorithm ALG using old training data consisting of initial sample data D_sample_initial, and then estimates the results of the network scan using the estimation algorithm ALG using a validation set (= consisting of one row of the subsets V _1' to V _8' shown in FIG. 62 ). Based on the estimated network scan results, the estimation means 17 executes a process of counting A, which is the number of correctly estimated expected events (= "normal") when an expected event (= "normal") occurs, B, which is the number of erroneously estimated expected events (= "normal") as unexpected events (= "poor communication quality") when an expected event (= "normal") occurs, and C, which is the number of erroneously estimated expected events (= "normal") as unexpected events (= "network congestion") when an expected event (= "normal") occurs, for all rows of the subsets V _1' to V _8' shown in FIG. 62 .

Since one row of subsets V _1' to V _8' shown in Fig. 62 has eight pairs {RTT _m' , CMbest _m' } of scan response delay RTT _m' and communication characteristic CMbest m _' , if the expected event (="normal") is correctly estimated, A = 8 and B = C = 0 are obtained. As a result, when counting A, B, and C is performed for all rows of subsets V _1' to V _8' shown in Fig. 62, if the expected event (="normal") is correctly estimated in all subsets, A = 64 and B = C = 0 are obtained.

The estimation means 17 uses the validation set (= consisting of one row of the subsets V _{1 '} to V _{8 '} shown in FIG. 62) to execute a process of estimating the results of the network scan by the estimation algorithm ALG for all rows of the subsets V _{1 '} to V _{8 '} shown in FIG. 62, counts A, B, and C, and assigns the counted A, B, and C to a confusion matrix table as shown in FIG. 65.

Then, the estimation means 17 obtains A as OE, B+C as OIE, and A+B+C as ECE.

Here, OE represents the total number of events that occurred as expected but were correctly estimated, ECE represents the total number of events that occurred as expected, and OIE represents the total number of events that occurred as expected but were incorrectly estimated as unexpected.

Then, the estimation means 17 calculates TP and FN based on the acquired OE, OIE, and ECE using the following formula.

By calculating TP and FN based on OE, OIE, and ECE using equation (25), TP and FN in the first row of the confusion matrix CFM shown in FIG. 64(a) are obtained. In this case, as shown in FIG. 64(a), FN is obtained as the combined value of the two elements "inadequate communication quality" and "network congestion" in the confusion matrix. In this way, TP and FN are obtained based on the estimated results of the network scan.

Then, the estimation means 17 calculates FP and TN based on the calculated TP and FN using the following formula.

The TP at the intersection of the "Normal" row and the "Normal" column shown in FIG. 64(a) is calculated by substituting the total number of correctly estimated "normal" events that occurred as expected, OE, and the total number of "normal" events that occurred as expected, ECE, into formula (25A).The FP at the intersection of the "Insufficient communication quality" row and the "Normal" column shown in FIG. 64(a) and the FP at the intersection of the "Network congestion" row and the "Normal" column are calculated by substituting the TP calculated by formula (25A) into formula (26A).

Furthermore, FN (=FN1) at the intersection of the "Normal" row and the "Insufficient communication quality" column shown in FIG. 64(a) is calculated by substituting the total number OIE of "Normal" events that occurred as expected but were erroneously estimated as "Insufficient communication quality" events that occurred unexpectedly, and the total number ECE of "Normal" events that occurred as expected, into formula (25B). Then, TN at the intersection of the "Insufficient communication quality" row and the "Insufficient communication quality" column shown in FIG. 64(a) and TN at the intersection of the "Network congestion" row and the "Insufficient communication quality" column are calculated by substituting FN (=FN1) calculated by formula (25B) into formula (26B).

Furthermore, FN (=FN2) at the intersection of the "Normal" row and the "Network Congestion" column shown in FIG. 64(a) is calculated by substituting the total number OIE of "Normal" events that occurred as expected but were erroneously estimated as "Network Congestion" events that occurred unexpectedly, and the total number ECE of "Normal" events that occurred as expected, into formula (25B). Then, TN at the intersection of the "Insufficient Communication Quality" row and the "Network Congestion" column shown in FIG. 64(a) and TN at the intersection of the "Network Congestion" row and the "Network Congestion" column are calculated by substituting FN (=FN2) calculated by formula (25B) into formula (26B).

When the estimation means 17 calculates TP, FN, TN, and FP using the method described above, it generates a confusion matrix CFM shown in FIG. 64(a) consisting of the calculated TP, FN, TN, and FP.

The estimation means 17 generates the confusion matrix CFM shown in (b) and (c) of FIG. 64 in a similar manner. In this case, since the result of the network scan is "inadequate communication quality" in the confusion matrix CFM shown in (b) of FIG. 64, in the column direction, "inadequate communication quality" is an expected event, and "normal" and "network congestion" are unexpected events. As a result, by the above-mentioned method, TP and FN in the second row of the confusion matrix CFM shown in (b) of FIG. 64 are calculated based on OE, OIE, and ECE, and then FP and TN in the first and third rows are calculated based on TP and FN.

In addition, in the confusion matrix CFM shown in FIG. 64(c), since the result of the network scan is "network congestion", in the column direction, "network congestion" is an expected event, and "normal" and "poor communication quality" are unexpected events. As a result, by the above-mentioned method, TP and FN in the third row of the confusion matrix CFM shown in FIG. 64(c) are calculated based on OE, OIE, and ECE, and then FP and TN in the first and second rows are calculated based on TP and FN.

In this way, the estimation means 17 acquires TP and FN from the estimation results of the network scan, calculates FP and TN based on the acquired TP and FN, and generates a confusion matrix CFM consisting of TP, FN, FP, and TN. In addition, the estimation means 17 calculates FN, FP, and TN by adding up multiple elements in the confusion matrix.

This method of generating the confusion matrix CFM differs from the general method of generating a confusion matrix, in which TP, FN, FP, and TN are obtained based on the observation of the subject for which the confusion matrix is to be generated, and a confusion matrix is generated.

As described above, the estimation means 17 trains the estimation algorithm ALG using old training data consisting of the initial sample data D_sample_initial, and then generates a confusion matrix CFM when estimating the results of the network scan by the estimation algorithm ALG using the validation set (= consisting of all rows of the subsets V _{1 '} to V _{8 '} shown in FIG. 62 ).

In addition, the estimation means 17 trains the estimation algorithm ALG using the tentative sample data D_sample_tentative_u, and then similarly estimates the results of the network scan using the estimation algorithm ALG using the validation set (= consisting of all rows of the subsets V1 _' to V8 _' shown in Figure 62), and generates a confusion matrix nCFM when the results of the network scan are estimated in the same manner as the confusion matrix CFM.

Furthermore, after generating the confusion matrix nCFM, the estimation means 17 estimates the result of the network scan by the estimation algorithm ALG using the test data (=consisting of all of the subsets TD ₁ to TD ₄ shown in FIG. 63), and generates a confusion matrix tCFM when the result of the network scan is estimated by the same method as the confusion matrix CFM.

In this case, the estimation means 17 estimates the results of the network scan using the test data (= consisting of one row of the subsets TD ₁ to TD ₄ shown in FIG. 63) with the estimation algorithm ALG, and executes the process of counting A, B, and C for all rows of the subsets TD ₁ to TD ₄ shown in FIG. 63 based on the estimated results of the network scan.

In one row of subsets TD ₁ to TD ₄ shown in Fig. 63, there are four pairs {RTT _m' , CMbest _m' } of scan response delay RTT _m' and communication characteristic CMbest m _' , so if the expected event (="normal") is correctly estimated, A = 4 and B = C = 0 are obtained. As a result, when counting A, B, and C is performed for all rows of subsets TD ₁ to TD ₄ shown in Fig. 63, if the expected event (="normal") is correctly estimated in all subsets, A = 32 and B = C = 0 are obtained. The same applies when the expected event is "poor communication quality" or "network congestion".

When the estimation means 17 generates the confusion matrices CFM, nCFM, and tCFM, it outputs the generated confusion matrices CFM, nCFM, and tCFM to the calculation means 16.

The calculation means 16 receives the confusion matrices CFM, nCFM, and tCFM from the estimation means 17. Then, the calculation means 16 uses the elements TP, TN, FP, and FN of the confusion matrices CFM, nCFM, and tCFM to calculate Precision, which is the ratio of the proportion actually classified as TP among the proportion classified as TP or FP, and Recall, which is the ratio of the proportion actually classified as TP among the proportion classified as TP or FN.

Precision is calculated using equation (27), and Recall is calculated using equation (28).

Then, after calculating Precision and Recall, the calculation means 16 calculates the F1 score _F1 by substituting the calculated Precision and Recall into the following equation.

The calculation means 16 calculates an F1 score F ₁ by equations (27) to (29) based on the TP, TN, FP, and FN of the confusion matrix CFM. The calculation means 16 also calculates an F1 score nF ₁ _u by equations (27) to (29) based on the TP, TN, FP, and FN of the confusion matrix nCFM. The calculation means 16 also calculates an F1 score tF ₁ by equations (27) to (29) based on the TP, TN, FP, and FN of the confusion matrix tCFM. The calculation means 16 then outputs the F1 scores F ₁ , nF ₁ _u, and tF ₁ to the estimation means 17.

The F1 score _F1 is an evaluation index EVD that represents the accuracy of the estimation result when the estimation algorithm ALG is trained using the initial sample data D_sample_initial and the network scan result is estimated by the estimation algorithm ALG (the estimation algorithm ALG after training) using the validation set (= consisting of all rows of the subsets V1 _' to V8 _' shown in Figure 62).

In addition, the F1 score nF ₁ _u is an evaluation index nEVD_u that represents the accuracy of the estimation result when the estimation algorithm ALG is trained using the tentative sample data D_sample_tentative_u and the network scan result is estimated by the estimation algorithm ALG (the estimation algorithm ALG after training) using the validation set (= consisting of all rows of the subsets V _1' to _V 8' shown in FIG. 62).

Furthermore, the F1 score _tF1 is an evaluation index tEVD that represents the accuracy of the estimation result when the network scan result is estimated by the estimation algorithm ALG (the estimation algorithm ALG after training) using the test data (= consisting of all _{rows of} the subsets TD1 to TD4 shown in FIG. 63).

In an embodiment of the present invention, the Matthews correlation coefficients MCC, nMCC_u, and tMCC may be used as the evaluation indices EVD, nEVD_u, and tEVD, respectively, instead of the F1 scores F ₁ , nF ₁ _u, and tF _1. In this case, the calculation means 16 calculates the Matthews correlation coefficient MCC by equation (30) based on the TP, TN, FP, and FN of the confusion matrix CFM, calculates the Matthews correlation coefficient nMCC_u by equation (30) based on the TP, TN, FP, and FN of the confusion matrix nCFM, and calculates the Matthews correlation coefficient tMCC by equation (30) based on the TP, TN, FP, and FN of the confusion matrix tCFM.

The Matthews correlation coefficient MCC calculated by formula (30) is "+1" when the best training data is used, "-1" when the worst training data is used, and "0" when neutral (neither "best" nor "worst") training data is used.

When the calculation means 16 calculates the Matthews correlation coefficients MCC, nMCC_u, and tMCC, it outputs the Matthews correlation coefficient MCC to the estimation means 17 as an evaluation index EVD, outputs the Matthews correlation coefficient nMCC_u to the estimation means 17 as an evaluation index nEVD_u, and outputs the Matthews correlation coefficient tMCC to the estimation means 17 as an evaluation index tEVD.

Upon receiving the evaluation index nEVD_u consisting of the F1 score nF ₁ _1 and the evaluation index EVD consisting of the F1 score F ₁ from the calculation means 16, the estimation means 17 determines whether the F1 score nF ₁ _1 is greater than the F1 score F ₁ .

When the estimation means 17 determines that the F1 score nF ₁ _1 is greater than the F1 score F ₁ , it sets the F1 score nF ₁ _1 to the F1 score F ₁ , and sets the tentative sample data D_sample_tentative_1 including the subset d ₁ when the F1 score nF ₁ _1 is obtained as the training data. Then, the estimation means 17 replaces the variables of the estimation algorithm ALG using the training data by a method described later. After that, the estimation means 17 trains the estimation algorithm ALG using the next subset d ₂ , estimates the result of the network scan by the estimation algorithm ALG (the estimation algorithm ALG after training) using the validation set (=consisting of all of the subsets V _{1 '} to V 8'), and generates a confusion matrix nCFM_2 when the result of the network scan is estimated. Then, the estimation means 17 outputs the generated confusion matrix nCFM_2 to the calculation means 16.

On the other hand, when the estimation means 17 determines that the F1 score nF ₁ _1 is not greater than the F1 score F ₁ , it does not set the F1 score nF ₁ _1 to the F1 score F ₁ , but trains the estimation algorithm ALG using tentative sample data D_sample_tentative_2 including the next subset d ₂ , estimates the result of the network scan with the estimation algorithm ALG (the trained estimation algorithm ALG) using the validation set (=consisting of all of the subsets V _1' to _{V 8'} ), generates a confusion matrix nCFM_2 when the result of the network scan is estimated, and outputs the confusion matrix nCFM_2 to the calculation means 16.

Thereafter, when the estimation means 17 receives the evaluation index nEVD_u consisting of the F1 score nF ₁ _2 from the calculation means 16, it determines whether or not the F1 score nF ₁ _2 is greater than the F1 score F ₁ .

When the estimation means 17 determines that the F1 score nF ₁ _2 is greater than the F1 score F ₁ , it sets the F1 score nF ₁ _2 to the F1 score F ₁ , and sets the tentative sample data D_sample_tentative_2 including the subset d ₂ when the F1 score nF ₁ _2 is obtained as the training data. Then, the estimation means 17 replaces the variables of the estimation algorithm ALG using the training data by a method described later. After that, the estimation means 17 trains the estimation algorithm ALG using the tentative sample data D_sample_tentative_3 including the next subset d ₃ , estimates the result of the network scan by the estimation algorithm ALG (the estimation algorithm ALG after training) using the validation set (=consisting of all of the subsets V _1' to V _8' ), and generates a confusion matrix nCFM_3 when the result of the network scan is estimated. Then, the estimation means 17 outputs the generated confusion matrix nCFM_3 to the calculation means 16.

On the other hand, when the estimation means 17 determines that the F1 score nF ₁ _2 is not greater than the F1 score F ₁ , it does not set the F1 score nF ₁ _2 to the F1 score F ₁ , but trains the estimation algorithm ALG using tentative sample data D_sample_tentative_3 including the next subset d ₃ , estimates the result of the network scan by the estimation algorithm ALG (the estimation algorithm ALG after training) using the validation set (=consisting of all of the subsets _V 1' to _{V 8'} ), and generates a confusion matrix nCFM_3 when the result of the network scan is estimated. Then, the estimation means 17 outputs the generated confusion matrix nCFM_3 to the calculation means 16.

Thereafter, the estimation means 17 repeatedly executes the above-mentioned operation for all of the subsets d ₃ to d _U using the F1 scores F ₁ and nF ₁ _u.

In addition, when the estimation means 17 receives an evaluation index EVD consisting of the Matthews correlation coefficient MCC and an evaluation index nEVD_1 consisting of the Matthews correlation coefficient nMCC_1 from the calculation means 16, it determines whether the Matthews correlation coefficient nMCC_1 is greater than the Matthews correlation coefficient MCC.

When the estimation means 17 determines that the Matthews correlation coefficient nMCC_1 is greater than the Matthews correlation coefficient MCC, it sets the Matthews correlation coefficient nMCC_1 to the Matthews correlation coefficient MCC, and sets the tentative sample data D_sample_tentative_1 including the subset d ₁ when the Matthews correlation coefficient nMCC_1 is obtained as the training data. Then, the estimation means 17 replaces the variables of the estimation algorithm ALG using the training data by a method described later. After that, the estimation means 17 trains the estimation algorithm ALG using tentative sample data D_sample_tentative_2 including the next subset d ₂ , estimates the result of the network scan by the estimation algorithm ALG using the validation set (=consisting of all of the subsets V _1' to V _8' ), and generates a confusion matrix nCFM_2 when the result of the network scan is estimated. Then, the estimation means 17 outputs the generated confusion matrix nCFM_2 to the calculation means 16.

On the other hand, when the estimation means 17 determines that the Matthews correlation coefficient nMCC_1 is not greater than the Matthews correlation coefficient MCC, it does not set the Matthews correlation coefficient nMCC_1 to the Matthews correlation coefficient MCC, but trains the estimation algorithm ALG using the tentative sample data D_sample_tentative_2 including the next subset _d2 , estimates the result of the network scan by the estimation algorithm ALG using the validation set (=consisting of all of the subsets V1 _' to V8 _' ), and generates a confusion matrix nCFM_2 when the result of the network scan is estimated. Then, the estimation means 17 outputs the generated confusion matrix nCFM_2 to the calculation means 16.

Then, when the estimation means 17 receives the evaluation index nEVD_2 consisting of the Matthews correlation coefficient nMCC_2 from the calculation means 16, it determines whether the Matthews correlation coefficient nMCC_2 is greater than the Matthews correlation coefficient MCC.

When the estimation means 17 determines that the Matthews correlation coefficient nMCC_2 is greater than the Matthews correlation coefficient MCC, it sets the Matthews correlation coefficient nMCC_2 to the Matthews correlation coefficient MCC, and sets the tentative sample data D_sample_tentative_2 including the subset d ₂ when the Matthews correlation coefficient nMCC_2 is obtained as the training data. Then, the estimation means 17 replaces the variables of the estimation algorithm ALG using the training data by a method described later. After that, the estimation means 17 trains the estimation algorithm ALG using tentative sample data D_sample_tentative_3 including the next subset d ₃ , estimates the result of the network scan by the estimation algorithm ALG using the validation set (=consisting of all of the subsets V _1' to V _8' ), and generates a confusion matrix nCFM_3 when the result of the network scan is estimated. Then, the estimation means 17 outputs the generated confusion matrix nCFM_3 to the calculation means 16.

On the other hand, when the estimation means 17 determines that the Matthews correlation coefficient nMCC_2 is not greater than the Matthews correlation coefficient MCC, it does not set the Matthews correlation coefficient nMCC_2 to the Matthews correlation coefficient MCC, but trains the estimation algorithm ALG using the tentative sample data D_sample_tentative_3 including the next subset _d3 , estimates the result of the network scan by the estimation algorithm ALG using the validation set (=consisting of all of the subsets V1 _' to V8 _' ), and generates a confusion matrix nCFM_3 when the result of the network scan is estimated. Then, the estimation means 17 outputs the generated confusion matrix nCFM_3 to the calculation means 16.

Thereafter, the estimation means 17 repeatedly executes the above-mentioned operation for all of the subsets d ₃ to d _U using the Matthews correlation coefficients MCC and nMCC_u.

[Estimated network scan results]
In training the above-mentioned estimation algorithm ALG, the estimation means 17 uses the initial sample data D_sample_initial and tentative sample data D_sample_tentative_1 to D_sample_tentative_U each including a subset d ₁ to d _U generated from the standard sample data D_similarity to estimate the result of the network scan by the following method.

In addition, in explaining the method of estimating the results of a network scan, the scan response delays {RTT _1' _known}_z to {RTT _M' _known}_z contained in each of the hypothetical sample data D_sample_tentative_1 to D_sample_tentative_U are denoted as "scan response delays RTT _m ", and the communication characteristic CMbest _m _unknown contained in each of the hypothetical sample data D_sample_tentative_1 to D_sample_tentative_U and the communication characteristics {CMbest _1' _known}_z to {CMbest _M' _known}_z contained in the standard sample data D_similarity are denoted as "communication characteristic CMbest".

The estimation means 17 receives the scan response delay RTT _m and the communication characteristic CMbest from the extraction means 15. Then, the estimation means 17 classifies the terminal devices having the communication characteristic CMbest into G groups CLS ₁ to CLS _G (G is an integer equal to or greater than 3) using an LDA (Linear Discriminant Analysis) algorithm.

The LDA algorithm classifies terminal devices having communication characteristics CMbest into G groups CLS ₁ to CLS _G so as to maximize the ratio of between-group variance and within-group variance, thus ensuring maximum separation between the groups CLS ₁ to CLS _G from each other.

The LDA algorithm spheres the data (terminal devices with communication characteristics CMbest) based on the common covariance and classifies the data (terminal devices with communication characteristics CMbest) into the closest group centroid in the transformed space.

To achieve this, the separability between different groups (also called "intergroup covariance"), which is the distance between the means of different groups, is calculated. Then, the distance between the mean and the sample of each group (also called "intragroup covariance") is calculated. This allows a low-dimensional space to be constructed that maximizes the intergroup covariance and minimizes the intragroup covariance. Then, in the constructed low-dimensional space, data (terminal devices having communication characteristics CMbest) are classified into the closest group centroid. The LDA algorithm is described in "T. Hastie, R. Tibshirani, and J. H. Friedman, "The elements of statistical learning: data mining, inference, and prediction," 2nd ed. Chapter 4. pp.106-118, New York: Springer, 2009." and is publicly known. Therefore, a detailed classification method using the LDA algorithm will be omitted.

Moreover, the number G of the groups CLS ₁ to CLS _G is set to the same number as the number of causes of the network scan results attributable to the communication channel with the terminal device among the estimated results of the network scan. That is, since the three results of "normal", "inadequate communication quality" and "network congestion" are causes attributable to the communication channel with the terminal device, the number G of the groups CLS ₁ to CLS _G is set to "3". As a result, the estimation means 17 classifies the terminal devices having the communication characteristic CMbest into three groups CLS ₁ to CLS ₃ using the LDA algorithm. Then, the group CLS ₁ is a "normal" group, the group CLS ₂ is a "inadequate communication quality" group, and the group CLS ₃ is a "network congestion" group.

When classifying terminal devices having communication characteristics CMbest into groups _CLS1 to _CLS3 , the estimation means 17 learns an LDA algorithm using scan data when the network scan is "network congested" (communication characteristics CMbest when "network is congested"), and classifies terminal devices having communication characteristics CMbest into three groups _CLS1 to _CLS3 using the learned LDA algorithm. This makes it possible to most accurately estimate that "network congestion" is the cause of the failure of the network scan.

The estimation means 17 uses Model1, Model2 and Model3 for estimating the results of a network scan. Model1 handles groups _CLS1 and _CLS2 , Model2 handles groups _CLS1 and _CLS3 , and Model3 handles groups _CLS2 and _CLS3 .

Then, when Model 1 is used, the estimation means 17 calculates a D score D_1 that discriminates between groups CLS ₁ and CLS ₂ , when Model 2 is used, calculates a D score D_2 that discriminates between groups CLS ₁ and CLS ₃ , and when Model 3 is used, calculates a D score D_3 that discriminates between groups CLS ₂ and CLS ₃ .

When the D scores D_1 to D_3 are calculated, a discriminant function δ _pq (x) expressed by the following equation is used.

A vector consisting of the scan response delay RTT _m and the communication characteristic CMbest is substituted for x in formula (31A). Then, x consists of the scan response delay RTT _m and the communication characteristic CMbest (optimum training data) when the result of the network scan is known, or the scan response delay RTT _m and the communication characteristic CMbest when the result of the network scan is unknown. Furthermore, the average μ _p of the communication characteristic shown in formula (31C) is substituted for μ _g in formula (31A) when calculating the discriminant function of group p, and the average μ q of the communication characteristic shown in formula (31D) is substituted for μ _g in formula (31A) when calculating the discriminant function of group q. Furthermore, the prior average Π _p of the communication characteristic shown in formula (31E) is substituted for the prior average Π _g in formula (31A) when calculating the discriminant function of group p, and the prior average Π q of the communication characteristic shown in formula (31F) is substituted for the prior average Π _g in formula (31A). The joint covariance matrix C _pq of formula (31A) is expressed by formula (31B). On the right side of formula (31B), the first term is a covariance matrix in group p, and the second term is a covariance matrix in group q. Then, the communication characteristics classified into group p (a vector consisting of scan response delay RTT _m and communication characteristics CMbest) are substituted into x _mp of formula (31B), and the communication characteristics classified into group q (a vector consisting of scan response delay RTT _m and communication characteristics CMbest) are substituted into x _mq of formula (31B).

In formulas (31B), (31C), and (31D), M _p is the total number of communication characteristics (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) classified into group p, M _q is the total number of communication characteristics (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) classified into group q, m _p is an argument indicating one communication characteristic (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) classified into group p, and m _q is an argument indicating one communication characteristic (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) classified into group q. In formulas (31E) and (31F), M is the total number of communication characteristics (vectors consisting of scan response delay RTT _m and communication characteristics CMbestt). Formula (31C) represents the average μ _p of the communication characteristics (vector consisting of scan response delay RTT _m and communication characteristic CMbest) classified into group p, and formula (31D) represents the average μ _q of the communication characteristics (vector consisting of scan response delay RTT _m and communication characteristic CMbest) classified into group q. Formula (31E) represents the prior average Π _p of the communication characteristics (vector consisting of scan response delay RTT _m and communication characteristic CMbest) classified into group p, and formula (31F) represents the prior average Π _q of the communication characteristics (vector consisting of scan response delay RTT _m and communication characteristic CMbest) classified into group q.

Equation (31B) represents a merged covariance matrix C pq obtained by merging two covariance matrices in two groups (two groups CLS ₁ and CLS ₂ ; two groups CLS ₁ and CLS ₃ ; or two groups CLS ₂ and CLS ₃ ). When calculating a merged covariance matrix C ₁₂ obtained by merging two covariance matrices in two groups CLS ₁ and CLS ₂ , in equation (31A), p represents group CLS ₁ , and q represents group CLS _2. Furthermore, when calculating a merged covariance matrix _{C 13 obtained} by merging two covariance matrices in two groups CLS ₁ and CLS ₃ , in equation (20A), p represents group CLS ₁ , and q represents group CLS ₃ . Furthermore, when calculating a merged covariance matrix _C23 by merging the two covariance matrices in the two groups _CLS2 and _CLS3 , p represents group _CLS2 and q represents group _CLS3 in formula (31A).

The joint covariance matrix C ₁₂ in Model 1 is expressed by the following equation.

In formula (32), _M1 corresponds to _Mp in formula (31B), M2 corresponds to _Mq in formula (31B), _m1 corresponds to _mp in formula (31B), and _m2 corresponds to _mq in formula (31B). Also, _μ1 is calculated by substituting _M1 for _Mp in formula (31C) and substituting the communication characteristic classified into group _CLS1 (a vector consisting of scan response delay RTT _{m and} communication characteristic CMbest) into _xmp in formula (31C), and _μ2 is calculated by substituting _M2 for _Mq in formula (31D) and substituting the communication characteristic classified into group _CLS2 (a vector consisting of scan response delay RTT _m and communication characteristic CMbest) into _xmq in formula (31D). The joint covariance matrix _C12 is represented by the sum of the covariance matrix in group _CLS1 and the covariance matrix in group _CLS2 .

The joint covariance matrix C ₁₃ in Model 2 is expressed by the following equation.

In formula (33), _M1 corresponds to _Mp in formula (31B), _M3 corresponds to _Mq in formula (31B), _m1 corresponds to _mp in formula (31B), and _m3 corresponds to _mq in formula (31B). _μ1 is calculated by substituting _M1 for _Mp in formula (31C) and substituting the communication characteristic classified into group _CLS1 (a vector consisting of scan response delay RTT _m and communication characteristic CMbest) into _xmp in formula (31C), and _μ3 is calculated by substituting _M3 for _Mq in formula (31D) and substituting the communication characteristic classified into group _CLS3 (a vector consisting of scan response delay RTT _m and communication characteristic CMbest) into _xmq in formula (31D). The joint covariance matrix _C13 is represented by the sum of the covariance matrix in group _CLS1 and the covariance matrix in group _CLS3 .

The joint covariance matrix C ₂₃ in Model 3 is expressed by the following equation.

In formula (34), _M2 corresponds to _Mp in formula (31B), _M3 corresponds to _Mq in formula (31B), _m2 corresponds to _mp in formula (31B), and _m3 corresponds to _mq in formula (31B). _μ2 is calculated by substituting _M2 for _Mp in formula (31C) and substituting the communication characteristic classified into group _CLS2 (a vector consisting of scan response delay RTT _m and communication characteristic CMbest) into _xmp in formula (31C), and _μ3 is calculated by substituting _M3 for _Mq in formula (31D) and substituting the communication characteristic classified into group _CLS3 (a vector consisting of scan response delay RTT _m and communication characteristic CMbest) into _xmq in formula (31D). The joint covariance matrix _C23 is represented by the sum of the covariance matrix in group _CLS2 and the covariance matrix in group _CLS3 .

When calculating the D score D_1 in Model 1, the discriminant function ^f1 in group CLS ₁ and the discriminant function ^f2 in group CLS ₂ are expressed by the following equations.

In formula (35A), the prior average Π ₁ is calculated by substituting M ₁ into M _p in formula (31E) and substituting the total number of communication characteristics (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) into M in formula (31E). In formula (35B), the prior average Π ₂ is calculated by substituting M ₂ into M _q in formula (31F) and substituting the total number of communication characteristics (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) into M in formula (31F). In addition, the average μ ₁ in formula (35A) and the average μ ₂ in formula (35B) are calculated by the method described in formula (32).

Furthermore, when calculating the D score D_2 in Model 2, the discriminant function ^f1 in group CLS ₁ and the discriminant function ^f2 in group CLS ₃ are expressed by the following equations.

In formula (36A), the prior average Π ₁ is calculated by substituting M ₁ into M _p in formula (31E) and substituting the total number of communication characteristics (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) into M in formula (31E). In formula (36B), the prior average Π ₂ is calculated by substituting M ₃ into M _q in formula (31F) and substituting the total number of communication characteristics (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) into M in formula (31F). In addition, the average μ ₁ in formula (36A) and the average μ ₃ in formula (36B) are calculated by the method described in formula (33).

Furthermore, when calculating the D score D_3 in Model 3, the discriminant function ^f1 in group CLS ₂ and the discriminant function ^f2 in group CLS ₃ are expressed by the following equations.

In formula (37A), the prior average Π ₂ is calculated by substituting M ₂ into M _p in formula (31E) and substituting the total number of communication characteristics (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) into M in formula (31E). In formula (37B), the prior average Π ₃ is calculated by substituting M ₃ into M _q in formula (31F) and substituting the total number of communication characteristics (vectors consisting of scan response delay RTT _m and communication characteristics CMbest) into M in formula (31F). In addition, the average μ ₂ in formula (37A) and the average μ ₃ in formula (37B) are calculated by the method described in formula (34).

The D score D_1 is expressed by the difference between the discriminant function ^f1 shown in equation (35A) and the discriminant function ^f2 shown in equation (35B), that is, ^f1 - ^f2 .

Furthermore, the D score D_2 is expressed by the difference between the discriminant function ^f1 shown in the formula (36A) and the discriminant function ^f2 shown in the formula (36B), that is, ^f1 - ^f2 .

Furthermore, the D score D_3 is represented by the difference between the discriminant function ^f1 shown in the formula (37A) and the discriminant function ^f2 shown in the formula (37B), that is, ^f1 - ^f2 .

Therefore, the D scores D_1 to D_3 can be calculated by using equations (31) to (37).

The estimation means 17 uses one of Model1 to Model3 to calculate the D score (one of D_1 to D_3).

Then, the estimation means 17 judges whether the D score is less than zero (=0). When the estimation means 17 judges that the D score is less than zero (=0), it increments the count value DS_count, which indicates the number of D scores less than zero (=0), by "1".

On the other hand, when the estimation means 17 determines that the D score is equal to or greater than zero (=0), it keeps the count value DS_count unchanged.

The estimation means 17 performs this process for all terminal devices that belong to one base station.

The estimation means 17 counts the count value DS_count of the D score, and then sets the Model for estimating the results of the network scan to Model2 or Model3 based on the count value DS_count.

More specifically, the estimation means 17 first sets Model to "1" (i.e., Model 1), and when the count value DS_count is zero (=0) or is greater than (N _terminal ×D_thres), sets the model Model to "2" (i.e., Model 2). Note that N _terminal is the total number of terminal devices belonging to one base station. Also, D_thres is set to a real number in the range of 0 to 1.

Furthermore, when the count value DS_count is not zero (=0) and is equal to or less than (N _terminal ×D_thres), the estimation means 17 sets the model Model to "3" (i.e., Model 3).

Then, when the model Model is not "1" (i.e., when the model Model is "2" or "3"), the estimation means 17 determines whether the count value DS_count is zero (=0).

When it is determined that the count value DS_count is zero (=0), the estimation means 17 determines that the scan data for the terminal device is "normal."

On the other hand, when it is determined that the count value DS_count is not zero (=0), the estimation means 17 determines whether or not the count value DS_count is smaller than (N _terminal ×D_thres).

Then, when it is determined that the count value DS_count is smaller than (N _terminal ×D_thres), the estimation means 17 estimates that the cause of the failure of the network scan for a terminal device having a D score smaller than "0" is "insufficient communication quality."

On the other hand, when it is determined that the count value DS_count is not smaller than (N _terminal × D_thres) (i.e., when it is determined that the count value DS_count is greater than or equal to (N _terminal × D_thres)), the estimation means 17 estimates that the cause of the failure of the network scan for all terminal devices belonging to the base station is "network congestion" (i.e., network congestion).

In addition, when it is determined that the count value DS_count is zero (=0) and further that there is a scan response from all ports of the terminal device, the estimation means 17 estimates that the network scan for the terminal device is "normal."

On the other hand, when it is determined that the count value DS_count is zero (=0) and further that a response has been received from some of the ports of the terminal device, the estimation means 17 estimates that the cause of the failure of the network scan for the terminal device is a "closed port."

Furthermore, when it is determined that the count value DS_count is zero (=0) and further that there is no scan response from all ports of the terminal device, the estimation means 17 estimates that the cause of the failure of the network scan for the terminal device is "no network connection."

The estimation algorithm ALG is an algorithm that estimates the results of a network scan using the method described above.

[Variable replacement in estimation algorithm]
When the F1 score nF 1 when the result of the network scan is estimated by the estimation algorithm ALG using a validation set (=consisting of all of the subsets V _{1 ′} to V _{U ′} ) generated from the standard sample data D_similarity is larger than the F1 score F ₁ , or when the Matthews correlation coefficient _{nMCC_u} when the result of the network scan is estimated by the estimation algorithm ALG using the validation set (=consisting of all of the subsets V _{1 ′} to V _{U ′} ) is larger than the Matthews correlation coefficient MCC, the estimation means 17 replaces the variables of the estimation algorithm ALG using tentative sample data D_sample_tentative_max including a subset d _max (any of the subsets d ₁ to d _{U) when the F1 score nF 1 determined to be larger than the F1 score F 1 is finally obtained, or replaces the variables of the estimation algorithm ALG using a subset d max (any of the subsets d 1 to d U ) when the Matthews} correlation coefficient nMCC_max determined to be larger than the Matthews correlation coefficient MCC is finally obtained. 1. The variables of the estimation algorithm ALG are replaced with tentative sample data D_sample_tentative_max including any of the _data D_sample_tentative_max, ...

As described in the above formula (31), x in formula (31A) is a vector consisting of the scan response delay RTT _m and the communication characteristic CMbest. Furthermore, μ _p in formula (31C) is the average μ p of the communication characteristics classified into group p (the vector consisting of the scan response delay RTT _m and the communication characteristic CMbest), and μ _q in formula (31D) is the average μ _q of the communication characteristics classified into group _q (the vector consisting of the scan response delay RTT _m and the communication characteristic CMbest). Furthermore, the prior average Π _p in formula (31E) is the prior average of the communication characteristics classified into group p (the vector consisting of the scan response delay RTT _m and the communication characteristic CMbest), and the prior average Π _q in formula (31F) is the prior average of the communication characteristics classified into group q (the vector consisting of the scan response delay RTT _m and the communication characteristic CMbest).

Here, the tentative sample data D_sample_tentative_max including data of size n (any of data d ₁ to d _U ) when an F1 score nF ₁ that is ultimately determined to be greater than the F1 score F ₁ is obtained, or a subset d _max (any of subsets d ₁ to d _U ) when a Matthews correlation coefficient nMCC_u that is ultimately determined to be greater than the Matthews correlation coefficient MCC is obtained, is denoted as "sample data D_sample_optimum".

The estimation means 17 substitutes the sample data D_sample_optimum for x in the formula (31A), calculates an average of a vector consisting of the scan response delay RTT _m and communication characteristic CMbest classified into group p among the scan response delay RTT _m and communication characteristic CMbest constituting the sample data D_sample_optimum as an average μ _p , calculates an average of a vector consisting of the scan response delay RTT m and communication characteristic CMbest classified into group q among the scan response delay RTT _m and communication characteristic CMbest constituting the sample data D_sample_optimum as an average μ _q , and calculates an average of a vector consisting of the scan response delay RTT _m and communication characteristic CMbest classified into group p among the scan response delay RTT _m and communication characteristic CMbest constituting the sample data D_sample_optimum as an average μ q. The prior average of a vector consisting of scan response delays RTT _m and communication characteristics CMbest classified into group _q among the scan response delays RTT _m and communication characteristics CMbest constituting the sample data D_sample_optimum is calculated as the prior average Π _{q ,} thereby replacing the variables of the estimation algorithm ALG with the sample data D_sample_optimum.

This improves the accuracy of estimation when estimating the results of a network scan using the estimation algorithm ALG.

As described above, the subsets d ₁ to d _U may be generated from the aggregated scan response delay {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3). Therefore, when the subsets d ₁ to d _U are generated based on the scan response delay {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3), the variables of the estimation algorithm ALG are replaced using the subsets d ₁ to d _U consisting of a part of the scan response delay {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3).

[Verification using test data]
After the model variables are replaced by the optimal training data, the estimation means 17 uses the test data (=consisting of all of the subsets TD ₁ to TD ₄ ) to estimate the results of the network scan with the model (estimation algorithm ALG) in which the model variables have been replaced, and generates a confusion matrix tCFM based on the estimated results of the network scan and outputs it to the calculation means 16.

The calculation means 16 receives the confusion matrix tCFM from the estimation means 17, calculates the F1 score _tF1 based on the elements TP, TN, FP, and FN of the received confusion matrix tCFM using equations (27) to (29), and outputs the calculated F1 score _tF1 to the estimation means 17.

The estimation means 17 receives the F1 score _tF1 from the calculation means 16 and determines whether the F1 score _tF1 is equal to or greater than the F1 score _F1 .

When the estimation means 17 determines that the F1 score tF ₁ is equal to or greater than the F1 score F ₁ , it approves the model (estimation algorithm ALG) in which the model variables have been replaced and the training data (optimum training data).

When it is determined that the F1 score _tF1 is not equal to or greater than the F1 score _F1 , the estimation means 17 does not approve the model (estimation algorithm ALG) and training data (optimum training data) in which the model variables have been replaced.

In this way, the estimation means 17 uses the test data (=consisting of all of the subsets TD ₁ to TD ₄ ) to estimate the results of a network scan by the model (estimation algorithm ALG) in which the model variables have been replaced, and verifies the model (estimation algorithm ALG) in which the model variables have been replaced and the training data (optimum training data) by confirming that the F1 score tF ₁ is equal to or greater than the F1 score F _1. That is, the estimation means 17 uses the test data (=consisting of all of the subsets TD ₁ to TD ₄ ) to verify that the model (estimation algorithm ALG) in which the model variables have been replaced can more accurately estimate the results of a network scan performed in a communication environment in which the results of the network scan are unknown, compared to the model (estimation algorithm ALG) before the model variables were replaced.

66 is a flow chart for explaining the operation of a network scan. When the operation of a network scan is started, the network scanner 11 obtains a scan list including addresses Add ₁ to Add _M and port numbers PN _r of terminal devices to be the targets of the network scan, and a scan schedule from the scan scheduler 18 (step S1).

Then, the network scanner 11 sets m=1 (step S2), and sets PN _r =1 (step S3).

Then, the network scanner 11 performs a network scan with or without monitoring background traffic (step S4). Then, the network scanner 11 obtains a scan response and a latency (step S5).

Then, the network scanner 11 determines whether PN _r =PN _r _MAX (the maximum value of the port number PN _r ) is satisfied (step S6).

When it is determined in step S6 that _PNr is not equal to _{PNr_MAX} , the network scanner 11 sets _PNr to _PNr +1 (step S7). After that, the series of operations proceeds to step S4, and steps S4 to S7 are repeatedly executed until it is determined in step S6 that _PNr is equal to _{PNr_MAX} .

Then, if it is determined in step S6 that PN _r =PN _r _MAX, the network scanner 11 determines whether or not m =M (step S8).

If it is determined in step S8 that m is not equal to M, the network scanner 11 sets m to m+1 (step S9). After that, the series of operations proceeds to step S3, and steps S3 to S9 are repeatedly executed until it is determined in step S8 that m is equal to M.

Then, when it is determined in step S8 that m=M, the network scanner 11 acquires communication characteristics (multiple communication characteristics such as scan response delay RTT _m ) from the scan response, and outputs the acquired communication characteristics and the scan response to the scan analysis manager 12. The scan analysis manager 12 receives the scan response and communication characteristics from the network scanner 11, and creates or updates the host information database 13 based on the received scan response and communication characteristics. In this case, the scan analysis manager 12 creates or updates the host information database 13 by storing the scan response and communication characteristics in correspondence table TBL1 or correspondence table TBL2.

Thereafter, the estimation means 17 reads out a plurality of communication characteristics such as the scan response delay RTT _m from the correspondence table TBL1 (or the correspondence table TBL2), and extracts the communication characteristic CMbest that is most suitable for estimating the result of the network scan based on the plurality of communication characteristics such as the read scan response delay RTT _m (step S10).

The estimation means 17 then trains the estimation algorithm ALG that estimates the results of the network scan, and selects training data (step S11). In this case, the estimation means 17 estimates the results of the network scan using the standard sample data D_similarity, and trains the estimation algorithm ALG by evaluating the accuracy of the estimated results. In addition, the estimation means 17 selects the sample data (sample data D_sample_optimum) when the accuracy of the estimated results is the highest, as the training data.

Then, the estimation means 17 estimates the results of a network scan performed in a communication environment where the results of the network scan are unknown, using the trained estimation algorithm ALG (step S12).

Then, the update means 14 reads out the scan response delay {RTT _m _known} (m=1 to M) from correspondence table TBL1 (or correspondence table TBL2), and reads out the scan response delays {RTT _m' _known}_1 (m'=1 to M'), {RTT _m' _known}_2 (m'=1 to M'), and {RTT _m' _known}_3 (m'=1 to M') from correspondence table TBL3. Then, the update means 14 and the calculation means 16 calculate baseline scan response delays {b RTT _{_ m} _ unknown } (m=1 to M), {b RTT _{_ m'} _ known } _ 1 (m'=1' to M'), {b RTT _{_ m'} _ known } _ 2 (m'=1' to M'), {b RTT _{_ m'} _ known } _ 3 (m'=1' to M') based on the scan response delays { _{RTT m} _ unknown } (m=1 to M), { _{RTT m'} _ known } _ 1 (m'=1' to M'), {RTT m' _ known } _ 2 (m'=1' to M'), and { _{RTT m'} _ known } _ 3 (m'=1' to _M '), respectively _. _known}_3 (m'=1' to M') is generated (step S13).

Then, the updating means 14 aggregates the scan response delays based on the baseline scan response delays {b _RTT _ _m _unknown} (m = 1 to M), {b _RTT _ _m' _known} _1 (m' = 1 to M'), {b _RTT _ _m' _known} _2 (m' = 1 to M'), and { _b RTT _ _m' _known} _3 (m' = 1 to M') to update the known scan response database (= correspondence table TBL3) (step S14).

Then, in step S11, the scan scheduler 18 receives from the estimation means 17 the estimated results of the network scan when accuracy is highest, creates a rescan list based on the received estimated results of the network scan (step S15), and outputs the created rescan list to the network scanner 11.

The reason for creating a rescan list is that if the result of a network scan is "insufficient communication quality" or "network congestion," running a network scan at a different time may result in the network scan being "normal."

After step S15, the network scanner 11 determines whether the rescan list is empty (step S16).

When it is determined in step S16 that the rescan list is not empty, the series of operations proceeds to step S2, and steps S2 to S16 are repeatedly executed until it is determined in step S16 that the rescan list is empty.

Then, in step S16, if it is determined that the rescan list is empty, the network scan operation ends.

Figures 67 and 68 are first and second flowcharts, respectively, for explaining the detailed operation of step S10 in Figure 66.

67, when it is determined in step S8 of Fig. 66 that m = M, the extraction means 15 calculates correlation coefficients r _ij for all communication characteristics of M terminal devices acquired by network scanning by the above-mentioned method (step S101), where i is an argument in the column direction in Figs. 41 to 44, and j is an argument in the row direction in Figs. 41 to 44.

Then, the extraction means 15 sets j=1 (step S102), and sets i=1 (step S103). After that, the extraction means 15 extracts a correlation coefficient r _ij between the communication characteristics f _i and f _j (step S104), and determines whether the extracted correlation coefficient r _ij is greater than a threshold value Corr _r (step S105).

When it is determined in step S105 that the correlation coefficient r _ij is not greater than the threshold value Corr _r , the extracting means 15 deletes the communication characteristics f _i and f _j (step S106).

On the other hand, when it is determined in step S105 that the correlation coefficient r _ij is greater than the threshold value Corr _r , the extracting means 15 stores the communication characteristics f _i and f _j (step S107).

Then, after step S106 or step S107, the extraction means 15 determines whether i=I (I is the maximum number of communication characteristics in the column direction in Figures 41 to 44) (step S108).

When it is determined in step S108 that i is not equal to I, the extraction means 15 sets i to i+1 (step S109). After that, the series of operations proceeds to step S104, and steps S104 to S109 are repeatedly executed until it is determined in step S108 that i is equal to I.

Then, if it is determined in step S108 that i=I, the extraction means 15 determines whether j=J (J is the maximum number of communication characteristics in the row direction in Figures 41 to 44) (step S110).

When it is determined in step S110 that j is not equal to J, the extraction means 15 sets j to j + 1 (step S111). After that, the series of operations proceeds to step S103, and steps S103 to S111 are repeatedly executed until it is determined in step S110 that j is equal to J.

Then, when it is determined in step S110 that j=J, the extraction means 15 calculates a multiple linear regression based on all the stored communication characteristics by the above-mentioned method, with all the stored communication characteristics as explanatory variables and the scan response delay RTT _m as a response variable, and obtains the contribution rate ^R2 .

Thereafter, the extraction means 15 determines whether or not the contribution rate ^R2 is smaller than a threshold value _Sr (step S113).

When it is determined in step S113 that the contribution rate ^R2 is smaller than the threshold value _Sr , the extraction means 15 sets the mode of the estimation algorithm ALG to "uncorrelated" (step S114).

On the other hand, when it is determined in step S113 that the contribution rate ^R2 is equal to or greater than the threshold value _Sr , the extraction means 15 sets the mode of the estimation algorithm ALG to "correlation" (step S115).

Then, after step S114 or step S115, the extraction means 15 saves the waiting time and communication characteristics in the network scan and the mode of the estimation algorithm ALG (step S116).

68, after step S116 in FIG. 67, the extracting means 15 calculates a plurality of scores Q of a plurality of communication characteristics having a correlation coefficient r _ij larger than a threshold value Corr _r , and selects K communication characteristics from the plurality of communication characteristics in descending order of score Q (step S117).

Then, the extraction means 15 uses the training data to perform learning of a classification algorithm (e.g., SVM-train) (step S118).

Thereafter, the extraction means 15 sets k=K (step S119). Subsequently, the extraction means 15 classifies the (k×M) sample vectors x ₁₁ to x _kM of the communication characteristics into two groups GP1 and GP2 using the trained classification algorithm (step S120).

Then, the extraction means 15 calculates k weights w ₁ to w _k of the k communication characteristics by using the equations (16), (18), and (19) (step S121).

Thereafter, the extraction means 15 calculates k rank weights R ₁ to R _k based on the k weights w ₁ to w _k using equation (20) (step S122).

Then, the extraction means 15 determines the sample of the communication characteristic to which the smallest rank weight R _{k_min} is assigned based on the k rank weights R ₁ to R _k as the communication characteristic to be deleted (step S123). After that, the extraction means 15 deletes the sample of the communication characteristic to which the smallest rank weight R _{k_min} is assigned from the sample vectors x ₁₁ to x _kM (step S124).

Then, the extraction means 15 determines whether the subset of communication characteristics is empty (step S125).

When it is determined in step S125 that the subset of communication characteristics is not empty, the extraction means 15 sets k=k-1 (step S126). That is, the extraction means 15 decreases the number of samples in the subset of communication characteristics by "1". After that, the series of operations proceeds to step S120, and steps S120 to S126 are repeatedly executed until it is determined in step S125 that the subset of communication characteristics is empty.

Then, when it is determined in step S125 that the subset of communication characteristics is empty, the extraction means 15 detects the last deleted communication characteristic as the communication characteristic CMbest _m (step S127).Then, the series of operations proceeds to step S11 in FIG.

After steps S120 to S126 are performed for the first time, steps S120 to S126 are repeatedly performed until the subset of communication characteristics is empty, thereby sequentially deleting samples of communication characteristics that are least important for estimating the results of a network scan. At this time, after deleting one sample of a communication characteristic, the remaining samples are reclassified into two groups GP1 and GP2 using the learned classification algorithm (see step S120). As described above, group GP1 is a group into which communication characteristics that are important for estimating the results of a network scan are classified, and group GP2 is a group into which communication characteristics that are not important for estimating the results of a network scan are classified. Therefore, the accuracy of determining whether or not each sample is a communication characteristic that is important for estimating the results of a network scan is improved, and the least important sample for estimating the results of a network scan can be deleted.

The process of detecting the communication characteristic CMbest _m in accordance with the flowcharts shown in FIGS. 67 and 68 constitutes a "detection process."

Moreover, the extraction means 15 that detects the communication characteristic CMbest _m in accordance with the flowcharts shown in FIGS. 67 and 68 constitutes a "detection means."

Figures 69 to 71 are first to third flowcharts, respectively, for explaining the detailed operation of step S11 in Figure 66.

Referring to FIG. 69, after step S10 in FIG. 66, the extraction means 15 detects the standard sample data D_standard_z (z = 1 to Z) used to estimate multiple known estimation results of the network scan from the correspondence table TBL3 stored in the host information database 13 (step S131), and outputs the detected standard sample data D_standard_z (z = 1 to Z) to the calculation means 16.

Then, the extraction means 15 detects sample data x _{i _unknown} acquired in a communication environment where the result of the network scan is unknown from the correspondence table TBL1 (or the correspondence table TBL2) stored in the host information database 13 (step S132). More specifically, the extraction means 15 detects the communication characteristic CMbest _m according to the flowcharts shown in Figures 67 and 68, and detects the scan response delay RTT _m associated with the communication characteristic CMbest _m from the correspondence table TBL1 (or the correspondence table TBL2), thereby detecting the sample data x _{i _unknown} . After that, the extraction means 15 outputs the detected sample data x _{i _unknown} to the calculation means 16.

The calculation means 16 receives the standard sample data D_standard_z (z = 1 to Z) and the sample data x _{i _unknown} from the extraction means 15. Then, the calculation means 16 calculates a probability density function PDF of the scan response delay {RTT _m' _known}_z (m' = 1' to M', z = 1 to Z) of the standard sample data D_standard_z (z = 1 to Z) and the scan response delay RTT _m _unknown (m = 1 to M) of the sample data x _{i _unknown} (step S133).

After that, the calculation means 16 calculates the mean absolute deviation W′ _unknown of the sample data x _{i _ unknown} (step S134).

Then, the calculation means 16 sets z=1 (step S135), and calculates the mode m(Mo)_known by equation (9) based on the probability density function of the standard sample data D_standard_z (step S136). In this case, when there are two most likely scan response delay values, the calculation means 16 substitutes the two most likely scan response delay values for _r1 and _r2 in equation (9) to calculate the mode m(Mo)_known, and when there are three or more most likely scan response delay values, the calculation means 16 detects the minimum and maximum values from the three or more most likely scan response delays, and substitutes the detected minimum and maximum values for _r1 and _r2 in equation (9) to calculate the mode m(Mo)_known.

Thereafter, the calculation means 16 calculates the relative sample data C(x _{i _known_z} ) by equation (22B) based on the sample data x _{i _known_z} and the mode m(Mo)_known_z (step S137), and calculates the mean absolute deviation W' _{known_z} by substituting the calculated relative sample data C(x _{i _known_z} ) into equation (22A) (step S138). That is, the calculation means 16 calculates the mean absolute deviation W' _{known_z} by equation (22) based on the sample data x _{i _known_z} and the mode m(Mo)_known_z.

Subsequently, the calculation means 16 calculates the difference S _Sz of the mean absolute deviations based on the mean absolute deviation W' _unknown and the mean absolute deviation W' _{known_z} by the formula (23) (step S139).

Then, the calculation means 16 calculates the similarity P _Sz based on the difference S _Sz of the mean absolute deviations by using the formula (24) (step S140).

Then, the calculation means 16 judges whether z=Z or not (step S141). When it is judged in step S141 that z=Z is not true, the calculation means 16 sets z=z+1 (step S142). Then, the series of operations proceeds to step S136, and steps S136 to S142 are repeatedly executed until it is judged in step S141 that z=Z.

If it is determined in step S141 that z=Z, the process proceeds to step S143 in FIG. 70.

In the flowchart shown in FIG. 69, when Z=3, steps S137 to S140 are executed as follows.

In step S137 of FIG. 69, when z=1, the calculation means 16 assigns the scan response delay {RTT _m' _known}_1 (m'=1' to M') to x _{i _known_z} , assigns the mode m(Mo)_known_1 to m(Mo)_known_z, and calculates the relative sample data C(x _{i _known_1} ) by the formula (22B); when z=2, the calculation means 16 assigns the scan response delay {RTT _m' _known}_2 (m'=1' to M') to x _{i _known_z} , assigns the mode m(Mo)_known_2 to m(Mo)_known_z, and calculates the relative sample data C(x _{i _known_2 )} by the formula (22B). ) is calculated, and when z=3, the scan response delay {RTT _m' _known}_3 (m'=1' to M') is substituted into x _{i _known_z} , and the mode m(Mo)_known_3 is substituted into m(Mo)_known_z to calculate the relative sample data C(x _{i _known_3} ) according to equation (22B).

Also, in step S138 of FIG. 69, when z=1, the calculation means 16 assigns the relative sample data C(x _{i _known —} 1 ) to C(x _{i _known —} z ) to calculate the mean absolute deviation W' _{known — 1} according to equation (22A); when z=2, the calculation means 16 assigns the relative sample data C(x _{i _known —} 2 ) to C(x _{i _known —} z ) to calculate the mean absolute deviation W' _{known —} 2 according to equation (22A); and when z=3, the calculation means 16 assigns the relative sample data C(x _{i _known — 3} ) to C(x _{i _known — z} ) to calculate the mean absolute deviation W' _{known — 3} according to equation (22A).

Furthermore, in step S139 of FIG. 69, when z=1, the calculation means 16 assigns the mean absolute deviation W' _{known_1} to W' _{known_z} , assigns the mean absolute deviation W' _unknown to W' _unknown , and calculates the difference S _S1 of the mean absolute deviations by equation (23); when z=2, the calculation means 16 assigns the mean absolute deviation W' _{known_2} to W' _{known_z} , assigns the mean absolute deviation W' _unknown to W' _unknown , and calculates the difference S _S2 of the mean absolute deviations by equation (23); when z=3, the calculation means 16 assigns the mean absolute deviation W' _{known_3} to W' _{known_z} , and calculates the difference S S3 of the mean absolute deviations by _equation (23); The difference S _{1 S 3} in the mean absolute deviation is calculated by substituting _unknown using equation (23).

Furthermore, in step S140 of FIG. 69, when z=1, the calculation means 16 substitutes the difference S _{2 S1} in the mean absolute deviations into S 2 _Sz to calculate the similarity P _{2 S1} according to formula (24), when z=2, substitutes the difference S _{2 S2} in the mean absolute deviations into S 2 _Sz to calculate the similarity P _{2 S2} according to formula (24), and when z=3, substitutes the difference S _{2 S3} in the mean absolute deviations into S 2 _Sz to calculate the similarity P _{2 S3} according to formula (24).

70, when it is determined in step S141 of FIG. 69 that z=Z, the calculation means 16 outputs the similarities P _S1 , P _S2 , . . . , P _SZ to the extraction means 15.

The extraction means 15 receives the similarities P _S1 , P _S2 , ..., P _SZ from the calculation means 16, detects the highest similarity P _{S_highest} among the received similarities P _S1 , P _S2 , ..., P _SZ , and detects the standard sample data D_similarity when the highest similarity P _{S_highest} is obtained (step S143). In this case, when there are multiple highest similarities P _{S_highest} , the extraction means 15 detects an arbitrary similarity P _{S_highest} among the multiple highest similarities P _{S_highest} , and detects the standard sample data D_similarity when the highest similarity P _{S_highest} is obtained.

Then, the extraction means 15 outputs the standard sample data D_similarity when the similarity P _{S_highest} is obtained to the estimation means 17 .

The estimation means 17 receives the standard sample data D_similarity from the extraction means 15. After that, the estimation means 17 sets the F1 score _F1 to "0" (step S144).

Then, the estimation means 17 divides the standard sample data D_similarity into training data and test data (step S145), and divides the training data into a training set and a validation set (step S146).

Then, the estimation means 17 generates initial sample data D_sample_initial for modeling using the above-mentioned method, and trains the estimation algorithm ALG using old training data consisting of the generated initial sample data D_sample_initial (step S147).

Then, the estimation means 17 estimates the results of the network scan by the estimation algorithm ALG (the estimation algorithm ALG after training) using the validation set (step S148), and generates a confusion matrix CFM based on the estimated results of the network scan and outputs it to the calculation means 16.

The calculation means 16 receives the confusion matrix CFM from the estimation means 17, calculates the F1 score _F1 using equations (27) to (29) based on the elements TP, TN, FP, and FN of the received confusion matrix CFM (step S149), and outputs the calculated F1 score _F1 to the estimation means 17.

The estimation means 17 receives the F1 score F ₁ from the calculation means 16. Then, the estimation means 17 divides the training set into subsets d ₁ to d _U (step S150). After that, the series of operations proceeds to step S151 in FIG.

71, after step S150 in FIG. 70, the estimation means 17 sets u=1 (step S151), and trains the estimation algorithm ALG using tentative sample data D_sample_tentative_u including the subset d _u (step S152).

Then, the estimation means 17 estimates the results of the network scan using the validation set (=consisting of all of the subsets V _{1 '} to V _{U '} ) with the estimation algorithm ALG (step S153), and generates a confusion matrix nCFM_u based on the estimated results of the network scan and outputs it to the calculation means 16.

The calculation means 16 receives the confusion matrix nCFM_u from the estimation means 17, calculates the F1 score nF ₁ _u based on the elements TP, TN, FP, and FN of the received confusion matrix nCFM_u using equations (27) to (29) (step S154), and outputs the calculated F1 score nF ₁ _u to the estimation means 17.

The estimation means 17 receives the F1 score nF ₁ _u from the calculation means 16. Then, the estimation means 17 determines whether or not the F1 score nF ₁ _u is greater than the F1 score F ₁ (step S155).

When it is determined in step S155 that the F1 score nF ₁ _u is not greater than the F1 score F ₁ , the series of operations proceeds to step S159.

On the other hand, when it is determined in step S155 that the F1 score nF ₁ _u is greater than the F1 score F ₁ , the estimation means 17 sets the F1 score nF ₁ _u to F ₁ (step S156).

Then, the estimation means 17 sets tentative sample data D_sample_tentative_u including the subset d _u when the F1 score nF ₁ _u is obtained as training data (step S157), and replaces the model variables (variables of the estimation algorithm ALG) with the training data by the above-mentioned method (step S158).

Then, when it is determined in step S155 that the F1 score nF ₁ _u is not greater than the F1 score F ₁ , or after step S158, the estimation means 17 determines whether or not u=U (step S159).

When it is determined in step S159 that u is not equal to U, the estimation means 17 sets u to u+1 (step S160). After that, the series of operations proceeds to step S152, and steps S152 to S160 are repeatedly executed until it is determined in step S159 that u is equal to U.

Then, when it is determined in step S159 that u = U, the estimation means 17 uses the test data (= consisting of all of the subsets TD ₁ to TD ₄ ) to estimate the results of the network scan with a model (estimation algorithm ALG) in which the model variables have been replaced (step S161), and generates a confusion matrix tCFM based on the estimated results of the network scan and outputs it to the calculation means 16.

The calculation means 16 receives the confusion matrix tCFM from the estimation means 17, calculates the F1 score _tF1 using equations (27) to (29) based on the elements TP, TN, FP, and FN of the received confusion matrix tCFM (step S162), and outputs the calculated F1 score _tF1 to the estimation means 17.

The estimation means 17 receives the F1 score _tF1 from the calculation means 16 and determines whether the F1 score _tF1 is equal to or greater than the F1 score _F1 (step S163).

When it is determined in step S163 that the F1 score _tF1 is not equal to _or greater than the F1 score F1, the series of operations proceeds to step S151, and steps S151 to S163 are repeatedly executed until it is determined in step S163 that the F1 score _tF1 is equal to or greater than the F1 score _F1 .

Then, in step S163, when it is determined that the F1 score _tF1 is equal to or greater than the F1 score _F1 , the estimation means 17 approves the model (estimation algorithm ALG) in which the model variables have been replaced and the training data (finally, tentative sample data D_sample_tentative_u including a subset d _u that satisfies _F1 < _{nF1_u} ) (step S164). Note that the tentative sample data D_sample_tentative_u approved in step S164 becomes the data sample D_sample_initial when step S11 in FIG. 66 is executed for the second or subsequent times.

Then, after step S164, the estimation means 17 estimates the result of the network scan by using the sample data x _unknown acquired by the network scan in the communication environment where the result of the network scan is unknown, with the model (estimation algorithm ALG) in which the model variables have been replaced (step S165). After that, the series of operations proceeds to step S12 in FIG.

In steps S144 and S149 in FIG. 70 and S154, S155, S156, S157, S162 and S163 in FIG. 71, Matthews correlation coefficients MCC, nMCC_u and tMCC may be used instead of the F1 scores F ₁ , nF ₁ _u and tF ₁ , respectively.

71 is executed for the first time, in step S163, if it is determined that the F1 score tF ₁ is not equal to or greater than the F1 score F ₁ , and the loop of steps S152 to S160 is executed for the second time, in steps S152 and S153, the estimation algorithm ALG in which the model variables are finally replaced with optimal training data when the loop of steps S152 to S160 is executed for the first time is used. As a result, when the loop of steps S152 to S160 is executed for the second time, the F ₁ score nF ₁ _u calculated in step S154 is greater than the F ₁ score nF ₁ _u calculated in step S154 when the loop of steps S152 to S160 is executed for the first time. Similarly, the _F1 score _{nF1_u} calculated in step S154 when the loop from step S152 to step S160 is executed a third, fourth, ... will be greater than the F1 score _{nF1_u} calculated in step _S154 when the loop from step S152 to step S160 is executed a second, third, ....

Therefore, as the number of times the loop from step S152 to step S160 is executed increases, the F1 score _{nF1_u} calculated in step S154 in the Rth (R is an integer equal to or greater than 2) execution of the loop from step S152 to step S160 becomes larger than the _F1 score nF1_u calculated in step S154 in the (R-1)th execution of the loop from step _S152 to step _S160 . Then, in step S158, by replacing the model variables with optimal training data, the estimation algorithm ALG approaches an estimation algorithm when it is estimated that the result of the network scan is a known estimation result (any of "normal", "inadequate communication quality", and "network congestion"). In other words, the estimation algorithm ALG approaches an optimal algorithm.

This increases the likelihood that in step S163, the _F1 score _tF1 will be greater than or equal to the _F1 score _F1 , and also increases the likelihood that in step S164, the model (estimation algorithm ALG) and training data in which the model variables have been replaced (finally, tentative sample data D_sample_tentative_u including a subset d _u that satisfies F ₁ < nF ₁ _u) will be approved.

Therefore, after it is determined in step S163 that the F1 score _tF1 is not greater than or equal to the F1 score _F1 , the estimation algorithm ALG approaches an optimal algorithm (an algorithm in which the estimation result of the network scan becomes a known estimation result (any of "normal", "poor communication quality", and "network congestion")) without changing the validation set and test data, so that in step S164, the model (estimation algorithm ALG) and training data in which the model variables have been replaced (finally, tentative sample data D_sample_tentative_u including a subset d _u that satisfies _F1 < _{nF1_u} ) are likely to be approved.

Steps S151 to S164 in FIG. 71 correspond to updating the estimation algorithm ALG, which estimates the results of a network scan performed in a communication environment where the results of the network scan are unknown, to approach the optimal algorithm. Here, the optimal algorithm is an algorithm that estimates the results of a network scan such that the estimated results when the results of the network scan are estimated become known estimated results.

The estimation means 17 that executes steps S151 to S164 in FIG. 71 constitutes an "algorithm update means" that updates the estimation algorithm ALG so that it approaches the optimal algorithm.

After the estimation algorithm ALG approaches the optimal algorithm, in step S165, the result of the network scan is estimated by the estimation algorithm ALG that has approached the optimal algorithm using sample data x _unknown acquired by a network scan in a communication environment where the result of the network scan is unknown, so that the result of the network scan can be accurately estimated.

69, the extraction means 15 detects standard sample data D_standard_z (z=1 to Z). Then, in steps S135 to S142, the calculation means 16 calculates Z similarities P _S1 to P _SZ between the Z standard sample data D_standard_1 to D_standard_Z and the sample data x _{i _unknown} .

Thereafter, in step S143 of FIG. 70, the extraction means 15 detects the highest similarity P _{S_highest} among the Z similarities P _S1 to P _SZ .

Therefore, by executing steps S131 to S143, the extraction means 15 detects the standard sample data D_similarity (= the standard sample data D_standard_z when the highest similarity P _{S_highest} is obtained) having the probability _density function most similar to the probability density function of the sample data x i _unknown from the Z standard sample data D_standard_1 to standard sample data D_standard_Z.

Thereafter, the estimation means 17 trains the estimation algorithm ALG using old training data (=initial sample data D_sample_initial) (see step S147), and the calculation means 16 uses the validation set to calculate the F1 score _F1 when the results of the network scan are estimated by the trained estimation algorithm ALG (see steps S148 and S149).

Furthermore, the calculation means 16 and the estimation means 17 execute steps S151 to S160 for all of the U pieces of provisional sample data D_sample_tentative_1 to D_sample_tentative_U, each of which includes the U subsets d ₁ to d _U , so that the estimation means 17 acquires the largest F1 score nF ₁ _highest among the U pieces of F1 scores nF ₁ _1 to nF ₁ _U. When it is determined in step S155 that the F1 score nF ₁ _u is greater than _the F1 score F ₁ , the F1 score nF ₁ _u greater than the F1 score F 1 is set as the F1 score F ₁ (see step S156), so that the F1 score F ₁ when step S155 is executed for the uth time is greater than the F1 score F ₁ when step S155 is executed for the (u-1)th time. As a result, the F1 score nF ₁ _u determined to be greater than the F1 score F ₁ by the u-th execution of step S155 is greater than the F1 score nF 1 _u determined to be greater than the F1 score F ₁ by the (u- ₁ )-th execution of step S155. Therefore, by executing steps S151 to S160 for all of U pieces of provisional sample data D_sample_tentative_1 to D_sample_tentative_U each including U subsets d ₁ to d _U , the largest F1 score nF ₁ _highest among the U F1 scores nF ₁ _1 to nF ₁ _U is obtained.

Then, the estimation means 17 sets the provisional sample data D_sample_highest including the subset d _{u _highest} when the highest F1 score nF ₁ _highest is obtained among the U F1 scores nF ₁ _1 to nF ₁ _U as the training data (see step S157), and replaces the model variables (variables of the estimation algorithm ALG) with the set training data (see step S158).

As a result, the F1 score nF ₁ _highest indicates the highest accuracy in estimating the results of the network scan, and the variables of the estimation algorithm ALG are replaced with the hypothetical sample data D_sample_highest which includes the subset d _{u _highest} when the F1 score nF ₁ _highest is obtained, so that the accuracy in estimating the results of the network scan can be maximized by estimating the results of the network scan using the estimation algorithm ALG having variables replaced with the hypothetical sample data D_sample_highest.

In addition, since the standard sample data D_standard_z (z = 1 to Z) consists of a scan response delay {RTT _m' _known}_z (m' = 1' to M', z = 1 to Z) and a communication characteristic {CMbest _m' _known}_z (m' = 1' to M', z = 1 to Z), the process in which the extraction means 15 detects the standard sample data D_similarity (= the standard sample data D_standard_z when the highest similarity P _{S_highest} is calculated) having a probability density function most similar to the probability density function of the sample data x _{i _unknown} from the Z standard sample data D_standard_1 to standard sample data D_standard_Z constitutes a "detection process". Furthermore, the scan response delay {RTT _m' _known}_similarity (m' = 1' to M') included in the standard sample data D_similarity constitutes a "similar scan response delay." Furthermore, the extraction means 15 that executes the detection process constitutes a "detection means." Furthermore, since each of M' and Z is an integer equal to or greater than 2, {RTT _m' _known}_z (m' = 1' to M', z = 1 to Z) included in the standard sample data D_standard_z (z = 1 to Z) constitutes "multiple sets of M' second scan response delays."

Moreover, since the sample data x _{i _unknown} is composed of a scan response delay RTT _m _unknown (m = 1 to M) and a communication characteristic CMbest _m _unknown (m = 1 to M), it constitutes “first sample data". Then, the extraction means 15 that detects the communication characteristic CMbest _m according to the flowcharts shown in Figures 67 and 68 and detects the scan response delay RTT _m combined with the communication characteristic CMbest _m from the correspondence table TBL1 (or correspondence table TBL2) to detect the sample data x _{i _unknown} constitutes “generation means".

Furthermore, the standard sample data D_similarity is made up of scan response delay {RTT _m' _known}_similarity (m' = 1' to M') and communication characteristic {CMbest _m' _known}_similarity (m' = 1' to M'), and therefore constitutes "similar standard sample data". The extraction means 15 that detects the standard sample data D_similarity constitutes the "generation means".

Furthermore, the F1 scores F ₁ , nF ₁ (=nF ₁ _1 to nF ₁ _U), tF ₁ or the Matthews correlation coefficients MCC, nMCC (=nMCC_1 to nMCC_U), and tMCC constitute an "evaluation index", the F1 score F ₁ or the Matthews correlation coefficient MCC constitutes a "first evaluation index", the F1 scores nF ₁ (=nF ₁ _1 to nF ₁ _U) or the Matthews correlation coefficient nMCC (=nMCC_1 to nMCC_U) constitute a "second evaluation index", and the F1 score tF ₁ or the Matthews correlation coefficient tMCC constitutes a "third evaluation index".

Furthermore, the estimation means 17 that replaces the variables of the estimation algorithm ALG with the training data (=the tentative sample data D_sample_tentative including the subset d _{u_highest} ) constitutes an "update means" that updates the estimation algorithm ALG with the first sample data.

Furthermore, steps S136 to S140 are performed for all z (=1 to Z), and the standard sample data D_similarity when the highest similarity P _{S_highest} is obtained in step S143 is detected by detecting M' scan response delays {RTT _m' _unknown}_1 to {RTT m' _unknown}_Z having a distribution most similar to the distribution of M scan response delays RTT _m _unknown based on the distribution of Z scan response delays {RTT _m' _known}_z (m'=1' to M', z=1 to Z) of the standard sample data _{D_standard_z} ( _z =1 to Z) and the distribution of the scan response delay RTT _m _unknown (m=1 to M) of the sample data x i _unknown. This corresponds to detecting {RTT _{1 '} _known}_z through {RTT _{M '} _known}_z as similar scan response delays from a plurality of sets (=Z sets) of M' scan response delays {RTT _{1 '} _known}_z through {RTT _{M '} _known}_z.

Furthermore, calculating the mean absolute deviation _W'unknown in step S134 constitutes the "calculation process".

Furthermore, calculating the similarity P _Sz through steps S136 to S140 constitutes a "calculation process."

Furthermore, executing step S152 constitutes executing a "first training", and executing step S147 constitutes executing a "second training". And the estimation means 17 that executes steps S147 and S152 constitutes a "training means".

Figure 72 is a flowchart explaining the detailed operation of step S134 in Figure 69.

72, after step S133 in FIG. 69, the calculation means 16 calculates the mode m(Mo)_unknown of the sample data x _{i_unknown} by equation (9) based on the probability density function of the sample data x _{i_unknown} (step S1341).

Thereafter, the calculation means 16 calculates the relative sample data C(x _{i _unknown} )_unknown by the formula (21B) based on the sample data x _{i _unknown} and the mode m(Mo)_unknown (step S1342), and substitutes the calculated relative sample data C(x _{i _unknown} )_unknown into the formula (21A) to calculate the mean absolute deviation _W'unknown (step S1343). That is, the calculation means 16 calculates the mean absolute deviation _W'unknown by the formula (21) based on the sample data x _{i _unknown} and the mode m(Mo)_unknown. Then, after step S1343, the series of operations proceeds to step S135 in FIG. 69.

Figures 73 to 75 are first to third flowcharts, respectively, for explaining the detailed operation of step S147 in Figure 70.

73, after step S 146 in FIG. 70, the estimation means 17 detects the scan response delay {RTT _m′ _known}_similarity and the communication characteristic {CMbest _m′ _known}_similarity included in the standard sample data D_similarity received from the extraction means 15 .

Then, the estimation means 17 sets b=1 (step S1471) and generates a discriminant function δ _g (x) using the LDA algorithm (step S1472). Subsequently, the estimation means 17 sets a model Model for estimating the results of the network scan to Model=1 (step S1473), sets m _b =1 (step S1474), and sets DS_count=0 (step S1475). Here, m _b represents a terminal device belonging to one base station b.

Then, the estimation means 17 calculates the D-score of the terminal device having the communication characteristic {CMbest _m' _known}_similarity by using the discriminant function δ _g (x) (step S1476).

Then, the estimation means 17 determines whether the D score is less than zero (0) (step S1477).

When it is determined in step S1477 that the D score is less than zero (0), the estimation means 17 sets DS_count = DS_count + 1 (step S1478). That is, the estimation means 17 increments the count value DS_count by "1".

Then, when it is determined in step S1477 that the D score is not smaller than zero (0), or after step S1478, the estimation means 17 determines whether or not m _b =N _terminal (step S1479).

When it is determined in step S1479 that m _b is not equal to N _terminal , the estimation means 17 sets m _b =m _b +1 (step S1480). After that, the series of operations proceeds to step S1476, and steps S1476 to S1480 are repeatedly executed until it is determined in step S1479 that m _b =N _terminal .

Then, when it is determined in step S1479 that m _b =N _terminal , the estimation means 17 determines whether or not Model = 1 (step S1481).

When it is judged in step S1481 that Model=1, the estimation means 17 judges whether or not DS_count=0, or whether or not DS_count>(N _terminal ×D_thres) (step S1482).

When it is determined in step S1482 that DS_count is not 0 and DS_count is not greater than (N _terminal ×D_thres), the estimation means 17 sets Model=3 (step S1483).

On the other hand, when it is determined in step S1482 that DS_count=0 or DS_count>(N _terminal ×D_thres), the estimation means 17 sets Model=2 (step S1484).

Then, after step S1483 or step S1484, the series of operations proceeds to step S1474.

On the other hand, if it is determined in step S1481 that Model is not 1, the process proceeds to step S1485 in FIG. 74.

Referring to FIG. 74, when it is determined in step S1481 that Model is not 1, the estimation means 17 determines whether DS_count is 0 or not (step S1485).

When it is determined in step S1485 that DS_count = 0, the estimation means 17 determines that the scan data for the terminal device is "normal" (step S1486).

On the other hand, when it is determined in step S1485 that DS_count is not 0, the estimation means 17 determines whether or not DS_count<(N _terminal ×D_thres) (step S1487).

When it is determined in step S1487 that DS_count<(N _terminal ×D_thres) is not satisfied, the estimation means 17 estimates that the cause of the failure of the network scan for all terminal devices belonging to base station b is "network congestion" (step S1488).

On the other hand, when it is determined in step S1487 that DS_count<(N _terminal ×D_thres), the estimation means 17 estimates that the cause of the failure of the network scan for a terminal device having a D score smaller than “0” is “insufficient SNR (insufficient communication quality)” (step S1489).

Then, after any of steps S1486, S1488, and S1489, the series of operations proceeds to step S1490 in FIG. 75.

Referring to FIG. 75, after any of step S1486, step S1488, and step S1489 in FIG. 74, the estimation means 17 sets m _b =1 (step S1490).

Then, the estimation unit 17 judges whether or not the result of the network scan for the terminal device m- _b has been estimated (step S1491).

When it is determined in step S1491 that the result of the network scan for the terminal device _mb has not been estimated, the estimation means 17 determines whether or not there is no scan response for all ports of the terminal device _mb (step S1492).

When it is determined in step S1492 that there is a scan response from some of the ports of the terminal device _mb , the estimation means 17 determines whether or not there is a scan response from all of the ports of the terminal device _mb (step S1493).

When it is determined in step S1493 that there is a scan response from all ports of the terminal device _mb , the estimation means 17 estimates that the network scan for the terminal device _mb is "normal" (step S1494).

On the other hand, when it is determined in step S1493 that there is no scan response from some ports of the terminal device _mb , the estimation means 17 estimates that the cause of the failure of the network scan for the terminal device _mb is "port closed" (step S1495).The estimation means 17 then stores the obtained waiting time in the host information database 13 (step S1496).

On the other hand, when it is determined in step S1492 that there is no scan response from all ports of the terminal device _mb , the estimation means 17 estimates that the cause of the failure of the network scan for the terminal device _mb is "no network connection" (step S1497).

Then, when it is determined in step S1491 that the result for the terminal device m _b has been estimated, or after any of steps S1494, S1496 and S1497, the estimation means 17 determines whether or not m _b =N _terminal (step S1498).

When it is determined in step S1498 that m _b is not equal to N _terminal , the estimation means 17 sets m _b =m _b +1 (step S1499). After that, the series of operations proceeds to step S1491, and steps S1491 to S1499 are repeatedly executed until it is determined in step S1498 that m _b =N _terminal .

Then, when it is determined in step S1498 that m _b =N _terminal , the estimation means 17 determines whether or not b =B (step S1500).

When it is determined in step S1500 that b is not equal to B, the estimation means 17 sets b to b+1 (step S1501). After that, the series of operations proceeds to step S1472 in FIG. 73, and steps S1472 to S1501 are repeatedly executed until it is determined in step S1500 that b is equal to B.

If it is determined in step S1500 that b=B, the process proceeds to step S148 in FIG. 70.

Note that the detailed operations of step S148 in FIG. 70 and steps S152, S153, S161, and S165 in FIG. 71 are also performed according to the flowcharts shown in FIG. 73 to FIG. 75.

When the flowcharts shown in Figures 73 to 75 are executed as detailed operations of step S148 in Figure 70, if the estimation means 17 determines in step S1500 that b = B, it generates a confusion matrix CFM and outputs the generated confusion matrix CFM to the calculation means 16.

When the flowcharts shown in Figures 73 to 75 are executed as detailed operations of step S153 in Figure 71, if the estimation means 17 determines in step S1500 that b = B, it generates a confusion matrix nCFM_u and outputs the generated confusion matrix nCFM_u to the calculation means 16.

Furthermore, when the flowcharts shown in Figures 73 to 75 are executed as detailed operations of step S161 in Figure 71, if the estimation means 17 determines in step S1500 that b = B, it generates a confusion matrix tCFM and outputs the generated confusion matrix tCFM to the calculation means 16.

Steps S1490 to S1499 in FIG. 75 are steps for estimating the results of a network scan (either "normal," "port closed," or "no network connection") for each of all terminal devices belonging to one base station.

Fig. 76 is a flowchart for explaining the detailed operation of step S1472 in Fig. 73. Note that Fig. 76 explains generation of the discriminant function δ _g (x) by a trained LDA algorithm trained using scan data when the network scan is "normal" (communication characteristics CMbest when the "network is congested")

76, after step S1471 in FIG. 73 or step S1501 in FIG. 75, the estimation means 17 classifies terminal devices having communication characteristics CMbest into G (G=3) groups CLS ₁ to CLS ₃ using the LDA algorithm (step S1472-1).

Then, the estimation means 17 calculates the averages μ ₁ , μ ₂ , and μ ₃ of each group using the formula (31C) (or the formula (31D)) (step S1472-2).

Thereafter, the estimation means 17 calculates the prior averages Π ₁ , Π ₂ , Π ₃ of each group using the equation (31E) (or the equation (31F)) (step S1472-3).

The estimation means 17 then calculates the covariance matrix C of each group using equation (31B) (step S1472-4).

Then, the estimation means 17 calculates the joint covariance matrix _C12 of Model1 by equation (32) (step S1472-5). The estimation means 17 also calculates the joint covariance matrix _C13 of Model2 by equation (33) (step S1472-6). Furthermore, the estimation means 17 calculates the joint covariance matrix _C23 of Model3 by equation (34) (step S1472-7).

The estimation means 17 then generates a discriminant function δ _pq (x) by equation (31A) (step S1472-8). The joint covariance matrices C ₁₂ , C ₁₃ , and C ₂₃ calculated in steps S1472-5 to S1472-7 are substituted for “C” in equation (31A).

In the flowchart shown in FIG. 76, the estimation means 17 classifies, for example, terminal devices having communication characteristics CMbest into three groups CLS ₁ to CLS ₃ in step S1472-1.

As described above, group CLS ₁ is a "normal" group, group CLS ₂ is a "poor communication quality" group, and group CLS ₃ is a "network congestion" group. Model 1 handles groups CLS ₁ and CLS ₂ , Model 2 handles groups CLS ₁ and CLS ₃ , and Model 3 handles groups CLS ₂ and CLS ₃ .

Therefore, when Model1 is used, the joint covariance matrix _C12 calculated in step S1472-5 is substituted for "C" in step S1472-8. When Model2 is used, the joint covariance matrix _C13 calculated in step S1472-6 is substituted for "C" in step S1472-8. Furthermore, when Model3 is used, the joint covariance matrix _C23 calculated in step S1472-7 is substituted for "C" in step S1472-8.

Furthermore, since the failure causes of "inadequate communication quality" and "network congestion" are failure causes caused by the communication channel with the terminal device, as described above, classifying the terminal devices having the communication characteristic CMbest into three groups CLS ₁ to CLS ₃ in step S1472-1 corresponds to "classifying into a first group indicating that the network scan is normal, and into second and third groups indicating that the network scan failed due to a cause caused by the communication channel with the terminal device." In this case, group CLS ₂ constitutes the second group, and group CLS ₃ constitutes the third group.

Figure 77 is a flowchart explaining the detailed operation of step S1476 in Figure 73.

Referring to FIG. 77, after step S1475 or step S1480 in FIG. 73, the estimation means 17 determines whether Model=1 (step S1476-1).

When it is determined in step S1476-1 that Model=1, the estimation means 17 substitutes the joint covariance matrix C12 calculated in step S1472-5 of Fig. ₇₆ for _C12 in formula (35A) to calculate a discriminant function ^f1 (step S1476-2). This discriminant function ^f1 is a discriminant function expressed using the joint covariance matrix _C12 obtained by combining the covariance matrix of the group _CLS1 (= _g1 ) and the covariance matrix of the group _CLS2 (= _g2 ), the mean _μ1 of the group _CLS1 (= _g1 ), and the prior mean _Π1 of the group _CLS1 (= _g1 ).

Thereafter, the estimation means 17 substitutes the joint covariance matrix _C12 calculated in step S1472-5 of Fig. 76 for _C12 in formula (35B) to calculate a discriminant function ^f2 (step S1476-3). This discriminant function ^f2 is a discriminant function expressed using the joint covariance matrix _C12 obtained by combining the covariance matrix of the group CLS1 (= _g1 ) and the covariance matrix of the group _CLS2 (= _g2 ), _the mean _μ2 of the group _CLS2 (= _g2 ), and the prior mean _Π2 of the group _CLS2 (= _g2 ).

On the other hand, if it is determined in step S1476-1 that Model is not 1, the estimation means 17 determines whether Model is 2 or not (step S1476-4).

When it is determined in step S1476-4 that Model=2, the estimation means 17 substitutes the joint covariance matrix _C13 calculated in step S1472-6 of Fig. 76 for _C13 in formula (36A) to calculate a discriminant function ^f1 (step S1476-5). This discriminant function ^f1 is a discriminant function expressed using the joint covariance matrix _C13 obtained by combining the covariance matrix of group _CLS1 (= _g1 ) and the covariance matrix of group _CLS3 (= _g3 ), the mean _μ1 of group _CLS1 (= _g1 ), _and the prior mean _Π1 of group CLS1 (= _g1 ).

Thereafter, the estimation means 17 substitutes the joint covariance matrix _C13 calculated in step S1472-6 of Fig. 76 for _C13 in formula (36B) to calculate a discriminant function ^f2 (step S1476-6). This discriminant function ^f2 is a discriminant function expressed using the joint covariance matrix _C13 obtained by combining the covariance matrix of the group _CLS1 (= _g1 ) and the covariance matrix of the group _CLS3 (= _g3 ), the mean _μ3 of the group _CLS3 (= _g3 ), and the prior mean _Π3 of the group _CLS3 (= _g3 ).

On the other hand, when it is determined in step S1476-4 that Model=2 is not satisfied, the estimation stage 17 substitutes the joint covariance matrix _C23 calculated in step S1472-7 of Fig. 76 for _C23 in equation (37A) to calculate a discriminant function ^f1 (step S1476-7). This discriminant function ^f1 is a discriminant function expressed using the joint covariance matrix _C23 obtained by combining the covariance matrix of group _CLS2 (= _{g2 )} and _the covariance matrix of group _CLS3 (= _g3 ), the mean _μ2 of group CLS2 (= _g2 ), and the prior mean _Π2 of group _CLS2 (=g2).

Thereafter, the estimation means 17 substitutes the joint covariance matrix _C23 calculated in step S1472-7 of Fig. 76 for _C23 in formula (37B) to calculate a discriminant function ^f2 (step S1476-8). This discriminant function ^f2 is a discriminant function expressed using the joint covariance matrix _C23 obtained by combining the covariance matrix of group _CLS2 (= _g2 ) and the covariance matrix of group _CLS3 (= _{g3 )} , the mean _μ3 of group _CLS3 (= _g3 ), and the prior mean _Π3 of group CLS3 (= _g3 ).

Then, after any of step S1476-3, step S1476-6, and step S1476-8, the estimation means 17 calculates D score=f ¹ −f ² (step S1476-9). After that, the series of operations proceeds to step S1477 in FIG.

In the flowchart shown in FIG. 77, Model=1 is set in step S1473 in FIG. 73, so when step S1476 in FIG. 73 is executed first, it is determined in step S1476-1 in FIG. 77 that Model=1, and after steps S1476-2 and S1476-3 are executed in sequence, the D score is calculated in step S1476-9 using Model1.

Then, after step S1476-9, steps S1476 to S1480 in FIG. 73 are repeatedly executed until it is determined in step S1479 in FIG. 73 that m _b =N _terminal , and then in step S1481 it is determined that Model = 1, and steps S1482 and S1483 are executed in sequence, or steps S1482 and S1484 are executed in sequence.

When steps S1482 and S1483 are executed in sequence, Model = 3 is set (see step S1483), so when step S1476 is executed after moving from step S1483 to step S1474, it is determined in step S1476-1 of FIG. 77 that Model = 1 is not true, and furthermore, in step S1476-4, it is determined that Model = 2 is not true, and after steps S1476-7 and S1476-8 are executed in sequence, in step S1476-9, the D score is calculated using Model 3.

On the other hand, if steps S1482 and S1484 in FIG. 73 are executed in sequence, Model=2 is set (see step S1484), and when step S1476 is executed after moving from step S1484 to step S1474, it is determined in step S1476-1 in FIG. 77 that Model=1 is not true, and further, in step S1476-4, it is determined that Model=2 is true, and after steps S1476-5 and S1476-6 are executed in sequence, in step S1476-9, the D score is calculated using Model2.

Therefore, the flowchart shown in Figure 77 is a flowchart for calculating the D score according to the set Model (either Model 1, Model 2, or Model 3).

In addition, in the flowcharts shown in Figures 73 to 75 for estimating the results of a network scan, after setting Model = 1 in step S1473, the D score is calculated in step S1476 because, by using Model 1, it is first possible to estimate whether the network scan is "normal" or "failed (= failed due to insufficient communication quality)", and when it is estimated that the network scan is "failed (= failed due to insufficient communication quality)", the cause of the network scan failure can be further estimated.

In addition, in step S1473, after Model=1 is set, D scores are calculated for all terminal devices belonging to one base station b using Model1, and the number of D scores smaller than "0" is counted as the count value DS_count (see steps S1476 to S1480).

Then, when step S1481 is executed for the first time, Model=1 is set, so in step S1481 it is determined that Model=1, and in steps S1482 to S1484, Model=2 or Model=3 is set.

Therefore, Model2 or Model3 is set according to the number of D scores (=DS_count) calculated using Model1. In other words, Model2 or Model3 is set based on Model1.

Then, in steps S1482 to S1484, after Model=2 or Model=3 is set, the D-score is calculated using Model2 or Model3, and the number of D-scores smaller than "0" is counted as a count value DS_count (see steps S1476 to S1480). More specifically, when Model=2 is set, the D-score is calculated using the communication characteristics CMbest classified into group _CLS1 , the communication characteristics CMbest classified into group _CLS3 , and the joint covariance matrix _C13 shown in formula (33), and when Model=3 is set, the D-score is calculated using the communication characteristics CMbest classified into group _CLS2 , the communication characteristics CMbest classified into group _CLS3 , and the joint covariance matrix C23 shown in formula ( ₃₄ ). Then, based on the calculated D scores, the number of D scores less than "0" is counted as a count value DS_count.

Then, in step S1481, it is determined that Model=1 is not satisfied, and in steps S1485 to S1497 shown in Figures 74 and 75, Model 2 or Model 3 is used to estimate the result of the network scan (either "normal," "poor communication quality," "network congestion," "port closed," or "no network connection").

When Model2 is set and the results of a network scan are estimated, the number of D scores calculated using Model2 that are smaller than "0" (= count value DS_count) is used.

In addition, when Model 3 is set and the results of a network scan are estimated, the number of D scores calculated using Model 3 that are smaller than "0" (= count value DS_count) is used.

In this embodiment of the invention, the estimation algorithm ALG is composed of steps S1471 to S1501 shown in Figures 73 to 75 (including the flowcharts shown in Figures 76 and 77).

Figure 78 is a flowchart explaining the detailed operation of step S149 in Figure 70.

Referring to FIG. 78, after step S148 in FIG. 70, the calculation means 16 receives OE, ECE, and OIE from the estimation means 17, and detects the number of OE, ECE, and OIE received (step S149-1).

Then, the calculation means 16 calculates TP using formula (25A) based on the number of OEs and the number of ECEs (step S149-2).

Then, the calculation means 16 calculates the FN using formula (25B) based on the number of OIEs and the number of ECEs (step S149-3).

The calculation means 16 then calculates FP using TP according to equation (26A) (step S149-4).

Then, the calculation means 16 calculates Precision using formula (27) based on the proportion classified as TP and the proportion classified as FP (step S149-5). The calculation means 16 also calculates Recall using formula (28) based on the proportion classified as TP and the proportion classified as FN (step S149-6).

Thereafter, the calculation means 16 substitutes Precision and Recall into the formula (29) to calculate the F1 score _F1 (step S149-7). Then, the series of operations proceeds to step S150 in FIG.

Note that the detailed operations of steps S154 and S162 in Fig. 71 are also executed according to the flowchart shown in Fig. 78. When the detailed operation of step S154 is executed, the calculation means 16 receives OE, ECE, and OIE from the estimation means 17, and sequentially executes steps S149-1 to S149-7 based on the received OE, ECE, and OIE to calculate the F1 score nF ₁ _u. When the detailed operation of step S162 is executed, the calculation means 16 receives OE, ECE, and OIE from the estimation means 17, and sequentially executes steps S149-1 to S149-7 based on the received OE, ECE, and OIE to calculate the F1 score tF ₁ .

Moreover, calculating the F1 scores F ₁ , nF ₁ _u, tF ₁ in accordance with steps S149-1 to S149-7 constitutes the "calculation process".

Figure 79 is a flowchart to explain the detailed operation of step S149 when the Matthews correlation coefficient MCC is calculated in step S149 of Figure 70.

The flowchart shown in FIG. 79 is the same as the flowchart shown in FIG. 78, except that steps S149-4 to S149-7 in the flowchart shown in FIG. 78 have been replaced with steps S149-4A and S149-5A.

Referring to FIG. 79, when the detailed operation of step S149 in FIG. 70 is started, the calculation means 16 sequentially executes steps S149-1 to S149-3 described above. Then, after step S149-3, the calculation means 16 calculates FP using TP according to equation (26A), and calculates TN using FN according to equation (26B) (step S149-4A).

Then, the calculation means 16 substitutes TP, TN, FP, and FN into equation (30) to calculate the Matthews correlation coefficient MCC (step S149-5A). After that, the series of operations proceeds to step S150 in FIG. 70.

When the Matthews correlation coefficient MCC is calculated in steps S154 and S162 in FIG. 71, the detailed operations of steps S154 and S162 are also executed according to the flowchart shown in FIG. 79. When the detailed operation of step S154 is executed, the calculation means 16 sequentially executes steps S149-1 to S149-3, S149-4A, and S149-5A to calculate the Matthews correlation coefficient nMCC_u. When the detailed operation of step S162 is executed, the calculation means 16 sequentially executes steps S149-1 to S149-3, S149-4A, and S149-5A to calculate the Matthews correlation coefficient tMCC.

Figures 80 and 81 are first and second flowcharts, respectively, for explaining the detailed operation of step S13 in Figure 66.

80, after step S12 in Fig. 66, the update means 14 sets S _known =1 (step S171). S _known is an argument representing one of the known estimation results, "normal", "inadequate communication quality" and "network congestion", and is made up of 1, 2 and 3. For example, S _known =1 represents "normal", S _known =2 represents "inadequate communication quality" and S _known =3 represents "network congestion".

After step S171, the update means 14 extracts the scan response delay in the known estimation result S _known of the network scan from the correspondence table TBL3 (step S172). In this case, when S _known = 1, the update means 14 extracts the scan response delay {RTT _m' _known}_1 (m' = 1' to M') associated with "normal" in the correspondence table TBL3, when S _known = 2, the update means 14 extracts the scan response delay {RTT _m' _known}_2 (m' = 1' to M') associated with "poor communication quality" in the correspondence table TBL3, and when S _known = 3, the update means 14 extracts the scan response delay {RTT _m' _known}_3 (m' = 1' to M') associated with "network congestion" in the correspondence table TBL3.

After step S172, the update means 14 judges whether or not S _known =S _{known_MAX} (step S173), where S _{known_MAX} is the total number of known estimation results, which is "3".

When it is determined in step S173 that S _known is not equal to S _{known_MAX} , the update means 14 sets S _known = S _known + 1 (step S174). After that, the series of operations proceeds to step S172, and steps S172 to S174 are repeatedly executed until it is determined in step S173 that S _known = S _{known_MAX} .

Then, if it is determined in step S173 that S _known = S _{known_MAX} , the update means 14 extracts the scan response delay {RTT _m _unknown} (m = 1 to M) of the network scan performed in a communication environment where the result of the network scan is unknown from the correspondence table TBL1 (or correspondence table TBL2) (step S175).

Thereafter, the update means 14 normalizes each of the scan response delays {RTT _m' _known}_1 (m' = 1' to M'), {RTT _m' _known}_2 (m' = 1' to M'), {RTT _m' _known}_3 (m' = 1' to M'), and {RTT _m _unknown} (m = 1 to M) (step S176).

Next, the update means 14 sets S _type = 1 (step S177). S _type is an argument that indicates one of the results of the network scan, "normal", "poor communication quality", "network congestion" and "unknown", and is made up of 1, 2, 3 and 4. For example, S _type = 1 indicates "normal", S _type = 2 indicates "poor communication quality", S _type = 3 indicates "network congestion", and S _type = 4 indicates "unknown". As a result, the total number of results of the network scan, S _{type_MAX} , is "4".

After step S177 or step S191, the update means 14 sets b=1 (step S178). b represents the base station.

Then, after step S178 or step S189, the update means 14 sets m _b =1 (step S179), where m _b represents a terminal device belonging to one base station b.

After step S179 or step S187, the update means 14 outputs the scan response delays {RTT _m' _known}_1 (m' = 1' to M'), {RTT _m' _known}_2 (m' = 1' to M'), {RTT _m' _known}_3 (m' = 1' to M'), and {RTT _m _unknown} (m = 1 to M) normalized in step S176 to the calculation means 16.

The calculation means 16 receives the normalized scan response delays {RTT _m' _known}_1 (m' = 1' to M'), {RTT _m' _known}_2 (m' = 1' to M'), {RTT _m' _known}_3 (m' = 1' to M'), and {RTT _m _unknown} (m = 1 to M) from the update means 14. Then, the calculation means 16 calculates the average μ _mb of the scan response delays x _mb,i of the terminal device _mb by equation (38) (step S180).

In equation (38), i consists of consecutive numbers representing each port number _{PNr of terminal device mb from} the minimum value to the maximum value of port number _PNr , _nmb represents the total number of ports of terminal device _mb , and xmb _,i represents each normalized scan response delay RTT.

After step S180, the calculation means 16 calculates the standard deviation σ _mb of the scan response delay x _mb,i of the terminal device _mb by equation (39) (step S181).

81, after step S181, the calculation means 16 calculates the score z _mb of the scan response delay x _mb,i of the terminal device _mb by equation (1) (step S182).

Then, the calculation means 16 calculates a lower limit L _{b_mb} of the scan response delay x _mb,i of the terminal device _mb using equation (2) (step S183), and calculates an upper limit U _{b_mb} of the scan response delay x _mb,i of the terminal device _mb using equation (3) (step S184).

Then, the calculation means 16 stores the lower limit L _{b — mb} and the upper limit U _{b — mb} as the baseline scan response delay b _{RTT — mb} (step S185).

After step S185, the calculation means 16 judges whether m _b =N _terminal or not (step S186), where N _terminal is the total number of terminal devices m _b .

When it is determined in step S186 that m _b is not equal to N _terminal , the calculation means 16 sets m _b =m _b +1 (step S187). After that, the series of operations proceeds to step S180, and steps S180 to S187 are repeatedly executed until it is determined in step S186 that m _b =N _terminal .

Then, when it is determined in step S186 that m _b =N _terminal , the calculation means 16 determines whether or not b =B (step S188), where B is the total number of base stations.

When it is determined in step S188 that b is not equal to B, the calculation means 16 sets b to b+1 (step S189). After that, the series of operations proceeds to step S179, and steps S179 to S189 are repeatedly executed until it is determined in step S188 that b is equal to B.

Then, when it is determined in step S188 that b=B, the calculation means 16 determines whether or not S _type =S _{type_MAX} (step S190).

When it is determined in step S190 that S _type is not equal to S _{type_MAX} , the calculation means 16 sets S _type =S _type +1 (step S191). After that, the series of operations proceeds to step S178, and steps S178 to S191 are repeatedly executed until it is determined in step S190 that S _type is equal to S _{type_MAX} .

Then, if it is determined in step S190 that S _type =S _{type_MAX} , the series of operations proceeds to step S14 in FIG.

In addition, in step S182, the calculation means 16 randomly selects one scan response delay x _mb,i from _{the n mb scan} response delays x _mb,i of terminal device mb and calculates one score z _mb , or selects each of the n _mb scan response delays x _mb,i of terminal device _mb and calculates n _mb scores z _mb .

When one score _zmb is calculated in step S182, the calculation means 16 calculates one lower limit _Lb in step S183 and calculates one upper limit _Ub in step S184. As a result, the calculation means 16 stores one baseline scan response delay _{bRTT_mb} consisting of one { _Lb , _Ub } in step S185.

On the other hand, when n _mb scores z _mb are calculated in step S182, the calculation means 16 calculates n mb lower limits L _b corresponding to the n _mb scores z _mb in step S183, and calculates n _mb upper limits U _b corresponding to the n _mb scores z _mb in step S184. As a result, in step S185, the calculation means 16 stores n _mb baseline scan response delays b RTT_1_mb to _b RTT_nmb_mb consisting of n _mb {L _{b —} 1 , U _{b — 1} }, {L _{b — 2} , U _{b — 2 }} , ..., {L _{b — nmb} , U b _{— nmb }} .

Alternatively, the calculation means 16 may calculate the score _zβ shown in Table 1 above in step S182, calculate the lower limit _Lb using equations (4) to (6) in step S183, and calculate the upper limit _Ub using equations (4), (5), and (7) in step S184. In this case, the calculation means 16 stores one baseline scan response delay _{bRTT_mb} consisting of one { _Lb , _Ub } in step S185.

In the flowcharts shown in FIG. 80 and FIG. 81, steps S180 to S187 are executed for all terminal devices m _b for the first time, whereby a baseline scan response delay b RTT _mb is generated for all terminal devices m _b connected to one base station b when S _type =1 (the result of the network scan is "normal"); steps S180 to S187 are executed for all terminal devices m _b for the second time, whereby a baseline scan response delay b _{RTT _mb} is generated for all terminal devices m b connected to one base station b when S _type =2 (the result of the network scan is "inadequate communication quality"); steps S180 to S187 are executed for all terminal devices m _b for the third time, whereby a baseline scan response delay b _{RTT _mb} is generated for all terminal devices m _b connected to one base station b when S _type =3 (the result of the network scan is "network congestion"); and steps S180 to S187 are executed for all terminal devices m b for the third time, whereby a baseline scan response delay b _{RTT _mb} is generated for all terminal devices m _b connected to one base station b when S type =3 (the result of the network scan is "network congestion"). _b , a baseline scan response delay bRTT_mb is generated for all terminal devices m _b connected to one base station b when S _type =4 (network scan result is " _unknown ").

In addition, by executing steps S179 to S189 for the first time for all base stations b, when S _type = 1 (the result of the network scan is "normal"), a baseline scan response delay b _{RTT _mb} is generated for all terminal devices m _b connected to all base stations b. By executing steps S179 to S189 for the second time for all base stations b, when S _type = 2 (the result of the network scan is "inadequate communication quality"), a baseline scan response delay b _{RTT _mb} is generated for all terminal devices m _b connected to all base stations b. By executing steps S179 to S189 for the third time for all base stations b, when S _type = 3 (the result of the network scan is "network congestion"), a baseline scan response delay b _{RTT _mb} is generated for all terminal devices m _b connected to all base stations b. By executing steps S179 to S189 for the fourth time for all base stations b, a baseline scan response delay b RTT _{_mb} is generated for all terminal devices m b connected to all base stations b. = 4 (network scan result is "unknown"), a baseline scan response delay _{bRTT_mb} is generated for all terminal devices m- _b connected to all base stations b.

Furthermore, steps S178 to S191 are executed for all S _types , whereby a baseline scan response delay b _{RTT_mb} is generated for all terminal devices m _b connected to all base stations b in all S _types . In other words, baseline scan response delays { _{bRTT_m'_known} }_1 (m'=1' to M'), { _{bRTT_m'_known} }_2 (m'=1' to M'), { _{bRTT_m'_known} }_3 (m'=1' to M'), and { _{bRTT_m'_unknown} } (m=1 to M) are generated from the scan response delays { _{RTT m'_known} }_1 (m'=1' to M'), { _{RTT m'_known} }_2 (m'=1' to M'), { _RTT m'_known}_3 ( _m '=1' to M'), and _{{ bRTT_m_unknown} } (m=1 to M), respectively.

Figures 82 and 83 are first and second flowcharts, respectively, for explaining the detailed operation of step S14 in Figure 66.

82, after step S13 in FIG. 66, the calculation means 16 sets S _type =1 (step S201).

Then, the calculation means 16 calculates the mode m (X _{bRTT_Type} ) of the baseline scan response delay X _{bRTT_Type} by equation (9) (step S202), where the baseline scan response delay X _{bRTT_Type} is the baseline scan response delay b _{RTT_mb} for all terminal devices m _b connected to all base stations b in one _Type .

Thereafter, the calculation means 16 calculates the relative sample C(q _{i — Type} ) of the baseline scan response delay X _{bRTT —} Type by equation (8B) (step S203).

Then, the calculation means 16 substitutes the relative sample C(q _{i — Stype} ) into the formula (8A) to calculate the mean absolute deviation W_S _type (step S204).

Then, the calculation means 16 judges whether or not S _type =S _{type_MAX} (step S205).

When it is determined in step S205 that S _type is not equal to S _{type _MAX} , the calculation means 16 sets S _type =S _type +1 (step S206). After that, the series of operations proceeds to step S202, and steps S202 to S206 are repeatedly executed until it is determined in step S205 that S _type is equal to S _{type _MAX} .

Then, when it is determined in step S205 that S _type =S _{type_MAX} , the calculation means 16 sets S _known =1 (step S207).

Subsequently, after step S207 or step S217, the calculation means 16 acquires the mean absolute deviation W_S _known of the scan response delay ({RTT _m' _known}_z (m'=1' to M', z=1 to 3) in the known estimation result S _known of the network scan (step S208), and acquires the mean absolute deviation W_S _unknown of the scan response delay {RTT _m _unknown} (m=1 to M) in the unknown estimation result S _unknown of the network scan (step S209).
Thereafter, the calculation means 16 calculates the square of the mean absolute deviation W_S _known (W_S _known ) ² and the square of the mean absolute deviation W_S _unknown (W_S _unknown ) ² (step S210).

Then, the calculation means 16 calculates the sum of the square (W_S _known ) ² and the square (W_S _unknown ) ² ((W_S _known ) ² + (W_S _unknown ) ² ) (step S211), and calculates the square of the difference between the mean absolute deviation W_S _known and the mean absolute deviation W_S _unknown (W_S _known - W_S _unknown ) ² (step S212).

Referring to FIG. 83, after step S212, the calculation means 16 calculates (W_S _known ) ² + (W_S _unknown ) ² - (W_S _known - W_S _unknown ) ² (step S213), and calculates (W_S _known ) ² + (W_S _unknown ) ² + (W_S _known - W_S _unknown ) ² (step S214).

Then, the calculation means 16 substitutes the calculation results in steps S213 and S214 into equation (11) to calculate the similarity rate b _{sim_Known} (step S215).

After that, the calculation means 16 judges whether or not S _known =S _{known_MAX} (step S216).

When it is determined in step S216 that S _known is not equal to S _{known_MAX} , the calculation means sets S _known = S _known + 1 (step S217). After that, the series of operations proceeds to step S208, and steps S208 to S217 are repeatedly executed until it is determined in step S216 that S _known = S _{known_MAX} .

Then, when it is determined in step S216 that S _known =S _{known_MAX} , the calculation means 16 outputs the calculated similarity rates b _{sim_Sknown} to the update means 14 .

The update means 14 receives a plurality of similarity rates b _{sim_Sknown} from the calculation means 16 and detects the largest similarity rate b sim_max among the received plurality of similarity rates b _{sim_Sknown} . In this case, when there are a plurality of largest similarity rates b _{sim_max} , the update means 14 detects an arbitrary largest similarity rate b _{sim_max} among the plurality _of largest similarity rates b _{sim_max} .

Then, the update means 14 selects the known estimation result S _known having the largest similarity rate b _{sim_max} to the unknown estimation result S _unknown , and determines that the unknown estimation result S _unknown is the known estimation result S _known corresponding to the scan response delay ({RTT _m' _known}_z (m' = 1' to M', z = 1 to 3)) having the largest similarity rate b _{sim_max} (step S218).

After that, the update unit 14 determines whether there are a plurality of similar unknown scan response delays {RTT _m _unknown} (m=1 to M) (step S219).

When it is determined in step S219 that there are multiple similar unknown scan response delays {RTT _m _unknown} (m = 1 to M), the update means 14 selects any one unknown scan response delay {RTT _m _unknown} (m = 1 to M) from the multiple similar unknown scan response delays {RTT _m _unknown} (m = 1 to M) (step S220).

Then, when it is determined in step S219 that there are not multiple similar unknown scan response delays {RTT _m _known} (m = 1 to M), or after step S220, the update means 14 aggregates the scan response delay (= scan response delay {RTT _m' _known}_z (m' = 1' to M', z = 1 to 3) in the known estimation result S _known having the largest similarity rate b _{sim _max} and the scan response delay {RTT _m _known} (m = 1 to M) in the _unknown estimation result S unknown (step S221).

Thereafter, the update means 14 updates the known _scan response database (=correspondence table TBL3) by storing the scan response delay {RTT m' _known_aggregation} _{_z} (m'=1' to M', z=1 to 3) aggregated in step S221 in place of the scan response delay (=scan response delay {RTT _{m' _known} }_z (m'=1' to M', z=1 to 3)) corresponding to the known estimation result S known having the largest similarity rate b sim_max (any of "normal", "inadequate communication quality" and "network congestion") in correspondence table TBL3 (step S222). Then, after step S222, the series of operations proceeds to step S15 in FIG. 66.

In the flowcharts shown in FIGS. 82 and 83, steps S208 to S217 are repeatedly executed until S _known =S _{known_MAX} , thereby obtaining a similarity rate b sim1 between the scan response delay {RTT _m _unknown} (m=1 to M) in the unknown estimation result S _unknown and the scan response delay {RTT m' _known}_1 (m'=1 to M') in the known estimation result S _known of "normal", a similarity rate b sim2 between the scan response delay {RTT _m _unknown} (m=1 to M) in the unknown estimation result S unknown and the scan response delay {RTT m' _known}_2 (m'=1 to M') in the known estimation result S _known of "insufficient communication quality", and a similarity rate b _sim3 between the scan response delay {RTT _m _unknown} (m=1 to M) in the _unknown estimation result S unknown and the scan response delay {RTT m' _known}_3 (m'=1 to M') in the known estimation result S known of "insufficient communication quality", and a similarity rate b _sim4 between the scan response delay {RTT _m _unknown} (m=1 to M) in the unknown estimation result S unknown and the scan response delay {RTT m' _known}_4 (m'=1 to M') in the A similarity rate b sim3 between the scan response delay {RTT _m — _unknown } (m=1 to M) in “unknown” and the scan response delay {RTT _m′ — known} — 3 (m′=1′ to M′) in “network congestion” which is the known estimation result _{S known} is calculated.

Then, the update means 14 detects the largest similarity rate b _{sim_max} among the three similarity rates b _sim1 , b _sim2 , b _sim3 , and aggregates the _scan response delay (any of scan response delay {RTT _m' _known} _1 (m' = 1' to M'), {RTT m' _known} _2 (m' = 1' to M'), {RTT _{m '} _known} _3 (m' = 1' to M')) when the largest similarity rate b sim_max is obtained and the scan response delay {RTT _m _unknown} (m = 1 to M) (see steps S218 to S221).

For example, when the largest similarity rate b _{sim_max} is the similarity rate b _sim1 , the update means 14 aggregates the scan response delay {RTT _m' _known}_1 (m' = 1' to M') and the scan response delay {RTT _m _unknown} (m = 1 to M) (see step S221), and updates the known scan response database by storing the aggregated scan response delay {RTT _m' _known_aggregation}_1 (m' = 1' to M') in place of the scan response delay {RTT _m' _known}_1 (m' = 1' to M') associated with "normal" in correspondence table TBL3 (see step S222).

Furthermore, when the largest similarity rate b _{sim_max} is the similarity rate b _sim2 , the update means 14 aggregates the scan response delay {RTT _m' _known} _2 (m' = 1' to M') and the scan response delay {RTT _m _unknown} _2 (m' = 1 to M) (see step S221), and updates the known scan response database by storing the aggregated scan response delay {RTT _m' _known_aggregation} _2 (m' = 1' to M') in place of the scan response delay {RTT _m' _known} _2 (m' = 1' to M') associated with "insufficient communication quality" in correspondence table TBL3 (see step S222).

Furthermore, when the largest similarity rate b _{sim_max} is the similarity rate b _sim3 , the update means 14 aggregates the scan response delay {RTT _m' _known} _3 (m' = 1' to M') and the scan response delay {RTT _m _unknown} _3 (m' = 1 to M) (see step S221), and updates the known scan response database by storing the aggregated scan response delay {RTT _m' _known_aggregation} _3 (m' = 1' to M') in place of the scan response delay {RTT _m' _known} _3 (m' = 1' to M') associated with "network congestion" in correspondence table TBL3 (see step S222).

The update means 14 that executes step S218 constitutes a "determination means" that determines that the unknown estimation result S _unknown of the network scan when the scan response delay {RTT _m _known} (m = 1 to M) is acquired is a known _estimation result (any of "normal", "inadequate communication quality", and "network congestion") corresponding to the scan response delay {RTT _m' _known}_z (m' = 1' to M', z = 1 to 3) having the largest similarity rate b sim _max.

Moreover, the update means 14 that detects the largest similarity rate b _{sim_max} among a plurality of similarity rates b _{sim_Sknown} (S _known =1 to S _{known_MAX} ) constitutes the "detection means."

As shown in FIG. 87 described later, by detecting the scan response delay {RTT _m' _known}_z (m' = 1' to M', z = 1 to 3) that is most similar to the scan response delay {RTT _m _known} (m = 1 to M) acquired in a communication environment where the result of a network scan is unknown, using a similarity measure using the similarity rate b _sim , it is shown that the result of a network scan when the scan response delay {RTT _m _known} (m = 1 to M) is acquired can be correctly identified as the known estimated result of a network scan when the scan response delay _{ RTT _m' _known}_z (m' = 1' to M', z = 1 to 3) that is most similar to the scan response delay {RTT m _known} (m = 1 to M) is acquired.

Therefore, the scan response delay {RTT _m' _known}_z (m'=1 _' to M', z=1 to 3) having the largest similarity rate b sim_max with the scan response delay {RTT _{m' _known} }_z (m'=1' to M', z=1 to 3) among the _scan response delays {RTT _m' _known} _{_1} (m'=1' to M'), {RTT m _{' _known}_2 (m'=1' to M'), and {RTT m'} _known}_3 (m'=1' to M') is detected, and the scan response delay {RTT _m By setting the network scan result when a scan response delay {RTT m _unknown} (m = 1 to M) is acquired, it is possible to suppress misidentification of the network scan result when a scan response delay {RTT _m _unknown} (m = 1 to M) is acquired. In other words, it is possible to suppress misidentification of the network scan result when a scan response delay {RTT _m _unknown} (m = 1 to M) is acquired and correctly identify it.

Fig. 84 is a diagram showing an example of a confusion matrix, where (a) in Fig. 84 shows the confusion matrix when the result of a network scan is estimated by the estimation algorithm ALG using a validation set (=consisting of all of the subsets V ₁ '-V _{8 '} ), and (b) in Fig. 84 shows the confusion matrix when the result of a network scan is estimated by the estimation algorithm ALG using test data (=consisting of all of the subsets TD ₁ -TD ₄ ).

Referring to (a) of Figure 84, when the result of the network scan is "normal" (known estimated result), when the result of the network scan is estimated by the estimation algorithm ALG using the validation set (=consisting of all of subsets V1 _' to V8 _' ), a confusion matrix consisting of, for example, TP=64, FN=0, TN=0 and FP=0 is obtained.

Referring to (b) of FIG. 84, when the result of the network scan is “normal” (known estimation result), when the result of the network scan is estimated by the estimation algorithm ALG using the test data (=consisting of all of the subsets TD ₁ to TD ₄ ), a confusion matrix consisting of, for example, TP=32, FN=0, TN=0, and FP=0 is obtained.

Therefore, in step S149 of FIG. 70 and step S154 of FIG. 71, the confusion matrix shown in FIG. 84(a) (the confusion matrix when the result of the network scan is estimated by the estimation algorithm ALG using the validation set (=consisting of all of the subsets V1 _' to V8 _' )) is generated, and in step S162 of FIG. 71, the confusion matrix shown in FIG. 84(b) (the confusion matrix when the result of the network scan is estimated by the estimation algorithm ALG using the test data (=consisting of all of the subsets _TD1 to _TD4 )) is generated.

The detailed operation of step S12 in FIG. 66 is performed according to the flowcharts shown in FIG. 73 to FIG. 75.

Fig. 85 is a schematic diagram showing the operation of the network scanning device 1. Referring to Fig. 85, scan response delays {RTT m' _known}_1 (m'=1' to M'), {RTT _m' _known}_2 (m'=1' to M'), {RTT _m' _known}_3 (m'=1' to M') are stored in correspondence table TBL3 corresponding to known estimated results of network scanning, "normal", "inadequate communication quality", _and "network congestion", and scan response delays {RTT _m _known} (m=1 to M) when the result of the network scanning is unknown are stored in correspondence table TBL1 (or correspondence table TBL2).

The extraction means 15 reads out the scan response delay {RTT _m _known} (m=1 to M) from the correspondence table TBL1 (or the correspondence table TBL2), and reads out the scan response delays {RTT _m' _known}_1 (m'=1 to M'), {RTT _m' _known}_2 (m'=1 to M'), and {RTT _m' _known}_3 (m'=1 to M') from the correspondence table TBL3. Then, the extraction means 15 outputs the read scan response delays {RTT _m _known} (m = 1 to M), {RTT _m' _known} _1 (m' = 1 to M'), {RTT _m' _known} _2 (m' = 1 to M'), and {RTT _m' _known} _3 (m' = 1 to M') to the calculation means 16.

The calculation means 16 calculates a similarity P S1 between the scan response delay {RTT _m _known} _1 (m'=1' to M') and the scan response delay {RTT _m _known} _2 (m'=1' to M'), a similarity P S2 between the scan response delay {RTT _m _known} _3 (m'=1' to M') and the scan response delay {RTT _m _known} _4 (m'=1' to M') based on the scan response delays {RTT _m _known} _1 (m'=1' to M'), {RTT _m' _known} _2 (m'=1' to M'), and {RTT _m' _known} _3 (m'=1' to M') received from the extraction means ₁₅ , using equations (21) to (24), respectively. Then, the similarity P _S2 between the scan response delay {RTT _{m '} _known}_3 (m'=1' to M') and the scan response delay {RTT _m _known}_4 (m'=1' to M') is calculated (see steps S131 to _S142 in FIG. 69).

The extraction means 15 detects the scan response delay {RTT _m' _known}_similarity (m'=1' to M') when the highest similarity is obtained among the similarities P _S1 to P _S3 calculated by the calculation means 16 (see step S143 in FIG. 70).

Here, the scan response delay {RTT _m _known} (m=1 to M) has the largest similarity b _{sim_max} with the scan response delay {RTT _m' _known} _2 (m'=1 to M'). As a result, the scan response delay {RTT _m' _known} _similarity_z (m'=1 to M', z=1 to 3) is composed of the scan response delay {RTT _m' _known} _2 (m'=1 to M'). Therefore, the extraction means 15 generates standard sample data D_similarity consisting of the scan response delay {RTT _m' _known} _2 (m' = 1' to M') and the communication characteristic {CMbest m' _known} _2 corresponding to the scan response delay {RTT _{m '} _known} _2 (m' = 1' to M'), and outputs it to the estimation means 17.

The estimation means 17 trains the estimation algorithm ALG using the standard sample data D_similarity received from the extraction means 15 (see steps S145 to S147 in FIG. 70).

On the other hand, the extraction means 15 extracts the communication characteristic {CMbest _m _unknown} that is most suitable for estimating the result of the network scan (see the flowcharts shown in Figs. 67 and 68).

Then, the estimation means 17 estimates the result of the network scan by the trained estimation algorithm ALG using the scan response delay {RTT _m — unknown} (m=1 to M) and the communication characteristic {CMbest _m — unknown} (see step S148 in FIG. 70).

The update means 14 reads out the scan response delays {RTT _m' _known} _ 1 (m' = 1' to M'), {RTT _m' _known} _ 2 (m' = 1' to M'), {RTT _m' _known} _ 3 (m' = 1' to M'), and {RTT _m _unknown} (m = 1 to M) and outputs them to the calculation means 16 .

The calculation means 16 calculates baseline scan response delays { _{b RTTm'} _known}_1 (m'=1' to M'), {b RTTm' _known}_2 (m'=1' to M'), {b RTTm _' _known}_3 (m'=1' to M'), {b RTTm' _known}_4 (m'=1' to M'), {b RTTm' _known}_5 (m'=1' to M'), {b RTTm _' _known}_6 (m'=1' to M'), {b RTTm _' _known}_7 (m'=1' to M'), {b RTTm' _known}_8 (m'=1' to M'), {b RTTm' _known}_9 (m'=1' to M'), {b RTTm' _known}_10 (m'=1' to M'), {b _RTTm' _known}_11 (m'=1' to M'), {b RTTm' _known}_2 (m'=1' to M'), {b RTTm' _known}_3 (m'=1' to M'), {b _RTTm' _known}_12 (m'=1' to M'), {b RTTm' _known}_13 (m'=1' to M'), {b RTTm' _known}_14 (m'=1' to M'), {b _{RTTm' _known}_15 (m'=1' to M'), {b RTTm'} _known}_26 (m'=1' to M'), {b _RTTm ' _known}_3 _unknown} (m=1 to M) is calculated (see the flowcharts shown in FIGS. 80 and 81).

Then, the calculation means 16 calculates the similarity rates b sim1 and b sim2 between the baseline scan response delay {b _{RTTm' _known} }_1 (m'=1' to M') and the baseline scan response delay {b _RTTm' _known}_2 (m'=1' to M'), {b _{RTTm '} _known}_3 (m'=1' to M'), and {b _RTTm _unknown} (m=1 to M) using equations ( ₈ ), (9), and ( ₁₁₎ . The similarity rate b _sim2 between the baseline scan response delay {b _RTTm _known}_2 (m' = 1' to M') and the baseline scan response delay {b _RTTm _known} (m = 1 to M), and the similarity rate b _sim3 between the baseline scan response delay {b RTTm' _known}_3 (m' = 1' to M') and the baseline scan response delay {b _RTTm _known} (m = 1 to M) are calculated (see steps S201 to S217 shown in Figures 82 and 83).

The update means 14 detects the largest similarity rate b _{sim_max} among the similarity rates b _sim1 to b _sim3 calculated by the calculation means 16. In this case, the update means 14 detects, for example, the similarity rate b _sim2 as the largest similarity rate b _{sim_max} .

Then, the update means 14 aggregates the scan response delay {RTT _m' _known} _2 (m' = 1' to M') and the scan response delay {RTT _m _unknown} (m = 1 to M) when _the similarity rate _b sim2 (= b sim _max ) is obtained by the above-mentioned method to generate the scan response delay {RTT _m' _known_aggregation} _2 (m' = 1' to M') (see step S221 in FIG. 83), and converts the generated scan response delay {RTT _m' _known_aggregation} _2 (m' = 1' to M') into the scan response delay {RTT _m' _known}_2 (m'=1' to M') is stored in correspondence table TBL3 to update correspondence table TBL3 (see step S222 in FIG. 83).

In FIG. 85, the similarity rate b _sim2 is the largest similarity rate b _{sim_max} . However, the scan response delay {RTT _m _unknown} (m=1 to M) for which the similarity rate b _sim1 or the similarity rate b _sim3 is the largest similarity rate b _{sim_max} may be obtained in a network scan performed in a communication environment in which the results of the network scan are unknown. Therefore, the scan response delays {RTT _{m' _known} _1 (m'=1 to M'), {RTT m'} _known} _2 (m'=1 to M'), {RTT _m' _known} _3 ( _m '=1 to M') stored in the correspondence table TBL3 by repeating the aggregation of the above-mentioned scan response delays RTT may be the aggregated scan response delays {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3) are sequentially updated.

The aggregated scan response delay {RTT _m' _known_aggregation}_z (m' = 1' to M', z = 1 to 3) is included in the standard sample data D_similarity together with the communication characteristics {CMbest _m' _known}_z (m' = 1' to M', z = 1 to 3) and is used to train the estimation algorithm ALG that estimates the results of a network scan. Therefore, this data is important for accurately estimating the results of a network scan performed in a communication environment where the results of the network scan are unknown.

Figure 86 is a flowchart explaining the method for producing standard sample data D_similarity.

Referring to FIG. 86, when production of the standard sample data D_similarity is started, a network scan is executed for M terminal devices in a communication environment in which the results of the network scan are unknown, and M scan response delays {RTT _{m_unknown} } (m=1 to M) of the network scan are obtained (step S301).

Then, when the results of a network scan performed in a communication environment where the results of a network scan are known are multiple known estimated results, F M' similarity rates between M' scan response delays {RTT _{m' _} known}_z (m' = 1' to M', z = 1 to 3) of F _M' (F _M' = 3) obtained and M scan response delays {RTT _{m _} unknown} (m = ₁ to M) are calculated using equations (21) to (24) (step S302).

Then, standard sample data is generated that includes _M' optimal communication characteristics and M' similar scan response delays corresponding to the detected M' similar scan response delays {RTT _m' _ known}_z (m' = 1' to M', z = 1 to 3) when the maximum similarity rate is obtained among the F M' similarity rates (step S303).

Then, the estimation algorithm ALG is trained using standard sample data (step S304).

Then, using the optimal communication characteristics corresponding to the M scan response delays {RTT _{m_unknown} } (m=1 to M) and the M scan response delays {RTT _{m_unknown} } (m=1 to M), the results of the network scan are estimated by the trained estimation algorithm ALG (step S305).

Then, it is determined whether the estimation result matches a known estimation result (step S306).

When it is determined in step S306 that the estimated result matches the known estimated result, M baseline scan response delays { _{bRTTm_unknown} } (m = 1 to M) are calculated based on the M scan response delays { _{RTTm_unknown} } (m = 1 to M) (step S307).

Then, for each of the M' scan response delays {RTT _m' _ known}_z (m' = 1' to M', z = 1 to 3) of the F _M' (F _M' = 3) sets, M' baseline scan response delays {b _RTT m' _ known}_z (m' = 1' to M', z = 1 to 3) of the F _M' sets are calculated, and M' baseline scan response delays {b _RTT m' _ known}_z (m' = 1' to M', z = 1 to 3) of the F M' sets are obtained (step S308).

Next, the calculation of the similarity rate between the M' baseline scan response delays {b _RTTm' _ known}_z (m' = 1' to M', z = 1 to 3) and the M scan response delays {RTT _m _ unknown} (m = 1 to M) using equation (11) is performed for all of the _M' baseline scan response delays {b _RTTm' _ known}_z (m' = 1' to M', z = 1 to 3) in the F M' sets to calculate the F _M' similarity rates (step S309).

Thereafter, the largest similarity ratio among the F _M' similarity ratios is detected (step S310).

Then, the M' scan response delays {RTT _{m'_known} }_z (m'=1' to M', z=1 to 3) when the highest similarity rate is obtained and the M scan response delays {RTT _{m'_unknown} } (m=1 to M) are aggregated to update the M' scan response delays {RTT _{m'_known} }_z (m'=1' to M', z=1 to 3) when the highest similarity rate is obtained (step S311).

Then, standard sample data is generated consisting of the optimal communication characteristics corresponding to the updated scan response delay and the updated scan response delay (step S312).

Then, in step S306, when it is determined that the estimation result does not match the known estimation result, or after step S312, the production of the standard sample data D_similarity is terminated.

The detailed operation of step S305 is performed according to the flowcharts shown in Figures 73 to 75.

In the flowchart shown in FIG. 86, an estimation algorithm ALG is trained using M' similar scan response delays RTT and M' optimal communication characteristics CMbest, and when an estimation result estimated by the trained estimation algorithm ALG matches a known estimation result, M' scan response delays {RTT _{m'_known} }_z (m'=1' to M', z=1 to 3) and M scan response delays {RTT _{m_unknown} } (m=1 to M) are aggregated to update M' scan response delays {RTT _{m'_known} }_z (m'=1' to M', z=1 to 3), and standard sample data consisting of the optimal communication characteristics corresponding to the updated scan response delays and the updated scan response delays is generated (see steps S307 to S312).

Therefore, it is possible to produce similar standard sample data that can accurately estimate known estimation results.

Furthermore, the flowchart shown in FIG. 86 is executed each time M scan response delays {RTT _m _unknown} (m=1 to M) are obtained when a network scan is performed in a communication environment where the results of the network scan are unknown. As a result, the scan response delays {RTT _m' _known}_1 (m' = 1' to M'), {RTT _m' _known}_2 (m' = 1' to M'), and {RTT _m' _known}_3 (m' = 1' to M') stored in correspondence table TBL3 are sequentially updated by the aggregated scan response delays {RTT _m' _known_aggregation}_1 (m' = 1' to M'), {RTT _m' _known_aggregation}_2 (m' = 1' to M'), and {RTT _m' _known_aggregation}_3 (m' = 1' to M'), respectively (see step S311).

The aggregated scan response delay {RTT _m' _known_aggregation}_z (m' = 1' to M', z = 1 to 3) is composed of M scan response delays {RTT _m _unknown} (m = 1 to M) obtained in a communication environment where the results of network scanning are unknown, and the scan response delay {RTT _{m ' _known}_z (m' = 1' to M', z = 1 to 3) that is most similar to the scan response delay {RTT m} _unknown} (m = 1 to M).

Then, the M' scan response delays in step S311 are composed of aggregated scan response delays {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3).

Therefore, the scan response delay {RTT _m' _known_aggregation}_z (m' = 1' to M', z = 1 to 3) aggregated when the flowchart shown in FIG. 86 is executed for the (n-1)th time (n is an integer of 2 or more) constitutes the standard sample data used to train the estimation algorithm ALG when M scan response delays {RTT _m _unknown} (m = 1 to M) are newly acquired when the flowchart shown in FIG. 86 is executed for the nth time (see step S304), and is used to replace the variables of the estimation algorithm ALG that estimates the results of a network scan, and is used to obtain the aggregated scan response delay {RTT _m' The estimation algorithm ALG in which variables are replaced by {_known_aggregation}_z (m' = 1' to M', z = 1 to 3) is used to estimate the results of a network scan performed in a communication environment where the results of the network scan are unknown (see step S305).

Then, finally, the aggregated scan response delay {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3) is used to generate similar standard sample data (see step S312).

Therefore, similar standard sample data is important for accurately estimating the results of network scans performed in communication environments where the results of network scans are unknown.

86 is executed each time M scan response delays {RTT _m _unknown} (m=1 to M) are acquired when a network scan is performed in a communication environment where the results of the network scan are unknown, so that the number of scan response delays RTTs when a network scan is performed for each terminal device m' increases in the aggregated scan response delays {RTT _m' _known_aggregation}_z (m'=1' to M', z=1 to 3). Therefore, the number of scan response delays RTTs included in the similar standard sample data can be increased, so that the estimation algorithm ALG can be trained by similar standard sample data including abundant scan response delays RTTs, and the estimation algorithm ALG can be quickly updated to an optimal algorithm.

In the flowchart shown in FIG. 86, steps S307 to S312 may be executed after step S305 without executing step S306.

[Performance evaluation using scan response delay]
A simulation will be described that evaluates the performance of detecting a scan response delay _RTT ( ₌ scan response delay {RTT _{m ' _known}_z (m' = 1' to M', z = any one of 1 to 3)) similar to a scan response delay {RTT m} _unknown} (m = 1 to M) obtained in a communication environment where the result of a network scan is unknown, using the similarity rate b sim using the baseline scan response delay b RTT. The simulation was performed using the NS3 simulator (version 3.30) described in Non-Patent Document 3.

In the simulation, the scan response delays {RTT _m' _known}_z (m' = 1' to M', z = 1 to 3) when the results of the network scan were "normal", "inadequate communication quality", and "network congestion" were obtained assuming a cellular system (LTE: Long Term Evolution).

Then, we evaluated the performance of a method that aggregates scan response delay RTTs to detect similar scan response delay RTTs, and compared the performance of a method that aggregates scan response delay RTTs to detect similar scan response delay RTTs with the performance of a method that does not aggregate scan response delay RTTs to detect similar scan response delay RTTs.

In this section on "Performance evaluation using scan response delays," the method of aggregating scan response delay RTTs to detect similar scan response delay RTTs is referred to as "this method," and the method of detecting similar scan response delay RTTs without aggregating scan response delay RTTs is referred to as "conventional method."

Note that in the "Performance evaluation using scan response delay" section, cases where the results of a network scan were "port closed" and "no network connection" were not evaluated.

The simulation parameters are shown in Table 2.

In Table 2, the propagation model is the "ITU Base coverage urban, Urban macro-cell scenario" described in Non-Patent Document 4.

In Table 2, set 1 was performed when the results of the network scan were known, and set 2 was performed when the results of the network scan were unknown.

The simulation aims to identify scan responses that can help infer the cause of a network scan failure (i.e., scan responses when a network scan is performed in a communication environment where the results of the network scan are unknown).
(I) Discrimination using mean absolute deviation calculated based on baseline scan response delay b _RTT Fig. 87 is a diagram showing the relationship between the similarity rate b _sim and the results of a network scan. In Fig. 87, (a) shows the relationship between the similarity rate b _{sim and the results of a network scan when using scan response 1 (Unknown scan response 1) when a network scan is performed in a communication environment where the results of the network scan are unknown, (b) shows the relationship between the similarity rate b sim} and the results of a network scan when using scan response 2 (Unknown scan response 2) when a network scan is performed in a communication environment where the results of the network scan are unknown, and (c) shows the relationship between the similarity rate b _sim and the results of a network scan when using scan response 3 (Unknown scan response 3) when a network _scan is performed in a communication environment where the results of the network scan are unknown.

Here, each of scan response 1 (Unknown scan response 1), scan response 2 (Unknown scan response 2), and scan response 3 (Unknown scan response 3) consists of M scan response delay RTTs.

87(a), (b), and (c), the vertical axis represents the similarity rate _bsim , and the horizontal axis represents the results of the network scan, which are "Normal", "Insufficient communication quality" (Low-SNR), and "Network congestion".

87A, when the similarity rate b _sim is calculated based on the scan response 1, the similarity rate b sim for "Normal" is 0.99, which is the maximum. Compared with the results of other network scans (i.e., "Insufficient communication quality" (Low-SNR) and "Network congestion"), the similarity rates _{b sim} are 0.36 and 0.81 higher. This indicates that the results of the network scan were correctly identified as "Normal".

87(b), when the similarity rate b _sim is calculated based on the scan response 2 (Unknown scan response 2), the similarity rate b _sim of "inadequate communication quality" (Low-SNR) is the maximum at 0.98, and compared with the results of other network scans (i.e., "Normal" and "Network Congestion"), the similarity rates b _sim are 0.21 and 0.60 higher. This indicates that the result of the network scan was correctly identified as "inadequate communication quality" (Low-SNR).

87(c), when the similarity rate b _sim is calculated based on the scan response 3 (Unknown scan response 3), the similarity rate b _sim of "Network Congestion" is the maximum at 0.89, and compared with the results of other network scans (i.e., "Normal" and "Low-SNR"), the similarity rates b _sim are 0.59 and 0.23 higher. This indicates that the result of the network scan was correctly identified as "Network Congestion".

Therefore, according to the identification results of this method using the baseline scan response delay _bRTT , it is possible to clearly identify the results of a network scan performed in a communication environment where the results of the network scan are unknown, using a scan response (Unknown scan response) obtained in the communication environment where the results of the network scan are unknown.

In order to confirm that the present method can achieve a large similarity rate b _sim when the sample size being compared (i.e., the number of scan response delays RTT) is different, the performance of the present method was evaluated when the number of ports in each terminal device was changed from "100 to 1000."

Fig. 88 is a diagram showing the relationship between the similarity rate b _sim and the number of ports that are scanned, in which (a) shows the relationship between the similarity rate b _sim and the number of ports that are scanned when scan response 1 is used, (b) shows the relationship between the similarity rate b _sim and the number of ports that are scanned when scan response 2 is used, and (c) shows the relationship between the similarity rate b _sim and the number of ports that are scanned when scan response 3 is used.

In addition, in (a), (b), and (c) of FIG. 88, the vertical axis represents the similarity rate b _sim , and the horizontal axis represents the number of ports to be scanned.

Furthermore, in (a) of FIG. 88, curve k4 shows the relationship between the similarity rate b _sim and the number of ports scanned when the result of the network scan is “Normal”, curve k5 shows the relationship between the similarity rate b _sim and the number of ports scanned when the result of the network scan is “Insufficient communication quality” (Low-SNR), and curve k6 shows the relationship between the similarity rate b _sim and the number of ports scanned when the result of the network scan is “Network Congestion”.

Furthermore, in (b) of FIG. 88, curve k7 shows the relationship between the similarity rate b _sim and the number of ports scanned when the result of the network scan is “Normal”, curve k8 shows the relationship between the similarity rate b _sim and the number of ports scanned when the result of the network scan is “Insufficient communication quality” (Low-SNR), and curve k9 shows the relationship between the similarity rate b _sim and the number of ports scanned when the result of the network scan is “Network Congestion”.

Furthermore, in (c) of FIG. 88, curve k10 shows the relationship between the similarity rate b _sim and the number of ports scanned when the result of the network scan is “Normal”, curve k11 shows the relationship between the similarity rate b _sim and the number of ports scanned when the result of the network scan is “Insufficient communication quality” (Low-SNR), and curve k12 shows the relationship between the similarity rate b _sim and the number of ports scanned when the result of the network scan is “Network congestion”.

Referring to (a) of FIG. 88, for all port numbers in the range of 100 to 1000, the network scan results when scan response 1 (Unknown scan response 1) is obtained have a similarity rate b _sim with “Normal” of 0.97 or more (see curve k4), and the similarity rate b _sim with “Insufficient communication quality” (Low-SNR) and “Network congestion” when scan response 1 (Unknown scan response 1) is obtained is lower than 0.97 (see curves k5 and k6).

Referring to (b) of FIG. 88, for all port numbers in the range of 100 to 1000, the result of the network scan when an Unknown scan response 2 is obtained has a similarity rate b _sim with “poor communication quality” (Low-SNR) of 0.92 or more (see curve k8), and the similarity rate b _sim with “Normal” and “Network Congestion” when an Unknown scan response 2 is obtained is lower than 0.92 (see curves k7 and k9).

Referring to (c) of FIG. 88, for all port numbers in the range of 100 to 900, the result of a network scan when an Unknown scan response 3 is obtained has a similarity rate b _sim with “Network Congestion” of 0.97 or more (see curve k12), and the similarity rate b _sim with “Normal” and “Poor communication quality” (Low-SNR) when an Unknown scan response 3 is obtained is lower than 0.97 (see curves k10 and k11). In addition, when the number of ports is 1000, the result of the network scan when scan response 3 (Unknown scan response 3) is obtained has a similarity rate b _sim of 0.89 with respect to “Network Congestion” (Congestion) (see curve k12), and the similarity rate b _sim with “Normal” and “Insufficient communication quality” (Low-SNR) when scan response 3 (Unknown scan response 3) is obtained is lower than 0.89 (see curves k10 and k11).

Therefore, it was confirmed that the present method can achieve a high similarity rate even when the sample size (the number of scan response delays RTT) is different.
(II) Comparison of the present method with conventional methods To demonstrate the effectiveness of the present method, we compared the present method with conventional methods. Most of the conventional methods, such as cosine similarity, triangle similarity (TS), sector area similarity (SS) and TT-SS, have similarities in the range of (0,∞), while the similarity rate of the present method is a real number in the range of (0,1), so the similarity rate was normalized in order to directly compare it with the present method.

Cosine similarity is a similarity measure described in Non-Patent Document 5, SS is a similarity measure described in Non-Patent Document 6, and TT-SS is a similarity measure described in Non-Patent Document 7 that combines TS and SS.

Figure 89 is a diagram showing a comparison between the similarity rate of the present method and the similarity rate of the conventional method. In Figure 89, (a) shows a comparison between the similarity rate of the present method and the similarity rate of the conventional method when scan response 1 (Unknown scan response 1) is obtained, (b) shows a comparison between the similarity rate of the present method and the similarity rate of the conventional method when scan response 2 (Unknown scan response 2) is obtained, and (c) shows a comparison between the similarity rate of the present method and the similarity rate of the conventional method when scan response 3 (Unknown scan response 3) is obtained.

In addition, in (a), (b), and (c) of Figure 89, "Baseline AAD" indicates the similarity rate of this method, "AAD" indicates the similarity rate calculated without using the baseline scan response delay _bRTT (i.e., using the scan response delay RTT), "Cosine" indicates the similarity rate of cosine similarity, "Triangle" indicates the similarity rate of TS, "Sector" indicates the similarity rate of SS, and "TS-SS" indicates the similarity rate of TS-SS.

89, the similarity rate of the present method is superior to that of the conventional method in all of scan response 1, scan response 2, and scan response 3. Unlike the similarity rate of the conventional method, the similarity rate of the present method can clearly distinguish all of "Normal", "Insufficient communication quality" (Low-SNR), and "Network congestion". Therefore, the similarity rate b _sim calculated using the baseline scan response delay b _RTT can accurately determine the largest similarity rate compared to the similarity rate b _sim calculated using the scan response delay RTT.

On the other hand, AAD is likely to fail to identify the network scan results because the similarity rates are close between "Normal", "Low-SNR" and "Congestion" in three samples (i.e., Unknown scan response 1, Unknown scan response 2 and Unknown scan response 3).

Similarly, as shown in (a) of FIG. 89, for cosine similarity, triangle similarity (TS), sector area similarity (SS) and TT-SS, the similarity rate b _sim is close between “Normal”, “Poor communication quality” (Low-SNR) and “Network congestion”, and there is a high possibility of failing to identify the results of the network scan.
(III) Estimation of the cause of failure of network scan To estimate the cause of failure of network scan, the performance of the estimation algorithm ALG was evaluated by using known scan responses as training data and setting D_thres to 0.5. Also, the scan response delay RTT and the minimum value of the scan response delay RTT were used as a combination to estimate the result of network scan.

Figure 90 illustrates the accuracy of estimating the results of a network scan. In Figure 90, the vertical axis represents the probability of correctly estimating the results of a network scan, and the horizontal axis represents the results of the network scan.

In addition, in FIG. 90, (a) shows the accuracy of estimating the results of a network scan when using conventional training data (training data including non-aggregated scan response delay RTTs), and (b) shows the accuracy of estimating the results of a network scan when using training data of the present method (training data including aggregated scan response delay RTTs).

Referring to (a) of Figure 90, when conventional training data was used, the network scan results were correctly estimated with a probability of more than 90% in the "Normal" and "Low-SNR" conditions, but in the "Network Congestion" condition, the network scan results were correctly estimated with an average probability of 73% and the network scan results were incorrectly estimated with a probability of 27%.

Referring to (b) of Figure 90, when the training data for this method was used, the results of the network scan could be correctly estimated with a probability of more than 90% in all cases of "Normal", "Poor communication quality" (Low-SNR) and "Network congestion".

Therefore, this method was found to be effective for estimating the results of network scans.

[Simulation of D-score]
Fig. 91 is a diagram showing a simulation result of the D score. Referring to Fig. 91, as a result of performing a simulation of the D score, it was found that when the network scan is "normal", the D score is larger than "0" in more than half of the terminal devices (see (a) of Fig. 91).

In addition, when the cause of a network scan failure was "insufficient communication quality," it was found that the D score was less than "0" in less than half of the terminal devices (see Figure 91 (b)).

Furthermore, when the cause of the network scan failure was "network congestion," it was found that the D score was smaller than "0" in more than half of the terminal devices (see (c) in Figure 91).

Therefore, by using the number of D scores less than "0" (= count value DS_count), it is possible to estimate whether the results of the network scan are "normal," "inadequate communication quality," or "network congestion."

Model 2 handles group CLS ₁ where the network scan is “normal” and group CLS ₃ where the network scan failed due to “network congestion”, and calculates a D-score D_2 to distinguish between group CLS ₁ and group CLS _3. If the number of D-scores D_2 that are smaller than “0” (=count value DS_count) among the calculated D-scores D_2 is “0”, it can be presumed that the network scan is “normal”, and if the number of D-scores D_2 that are smaller than “0” (=count value DS_count) is greater than half of the terminal devices, it can be presumed that the network scan failed due to “network congestion”.

Model 3 handles group CLS ₂ in which the network scan has failed due to “inadequate communication quality” and group CLS _{3 in which the network scan has failed due to “network congestion”, and calculates a D-score D_3 to distinguish between group CLS 2 and} group CLS _3. If the number of D-scores D_3 less than “0” (=count value DS_count) among the calculated D-scores D_3 is less than half of the terminal devices, it can be assumed that the network scan has failed due to “inadequate communication quality”, and if the number of D-scores D_3 less than “0” (=count value DS_count) is greater than half of the terminal devices, it can be assumed that the network scan has failed due to “network congestion”.

Therefore, first, in step S1485 of FIG. 74, it is determined whether DS_count = 0 or not. If it is determined that DS_count = 0 is not true, it is further determined whether DS_count < (N _terminal x D_thres) or not. If DS_count < (N _terminal x D_thres), it is presumed that the network scan failed for terminal devices having a D score smaller than "0" due to "insufficient communication quality". If DS_count < (N _terminal x D_thres) is not true (i.e., when DS_count is equal to or greater than (N _terminal x D_thres)), it is presumed that the network scan failed for all terminal devices belonging to the base station due to "network congestion" (see steps S1485, S1487 to S1489).

When Model 2 is set, in steps S1485, S1487 to S1489, the D-score D_2 is used to distinguish between group CLS ₁ in which the network scan is “normal” and group CLS ₃ in which the network scan has failed due to “network congestion”, so that, depending on the number of D-scores D_2 that are smaller than “0” (=count value DS_count), it is possible to accurately estimate whether DS_count=0 (i.e., the network scan is “normal”) or DS_count≠0 (i.e., the network scan has failed due to “inadequate communication quality” or “network congestion”). Then, when DS_count is not 0 (i.e., DS_count ≠ 0), depending on whether the count value DS_count, which is the number of D scores D_2 smaller than "0", is smaller than (N _terminal × D_thres), it can be accurately estimated that the cause of the failure of the network scan is "insufficient communication quality" or "network congestion".

On the other hand, when Model 3 is set, in steps SS1485, S1487 to S1489, a D-score D_3 is used to distinguish between group CLS ₂ in which the network scan has failed due to "inadequate communication quality" and group CLS ₃ in which the network scan has failed due to "network congestion". Therefore, when DS_count is not 0 (i.e., DS_count ≠ 0), it is possible to accurately estimate that the cause of the network scan failure is "inadequate communication quality" or "network congestion" depending on whether the count value DS_count, which is the number of D-scores D_3 smaller than "0", is smaller than (N _terminal ×D_thres). In addition, since the determination of whether DS_count=0 or not is clear, it is possible to accurately estimate whether DS_count=0 (i.e., the network scan is “normal”) or whether DS_count≠0 (i.e., the network scan has failed due to “inadequate communication quality” or “network congestion”).

Therefore, whether Model 2 or Model 3 is used, it is possible to accurately estimate whether a network scan is "normal" or "failed." When it is estimated that a network scan is "failed," it is also possible to accurately estimate that the cause of the network scan failure is "insufficient communication quality" or "network congestion."

Furthermore, when it is determined in step S1485 that DS_count = 0, the network scan is not immediately assumed to be "normal", but is instead assumed to be either "normal", a failure due to a "closed port", or a failure due to "no network connection" for each terminal device port, depending on the number of ports for which there was a scan response (see steps S1492 to S1495 and S1497). Therefore, by detecting the presence or absence of a scan response at each port, it is possible to accurately estimate whether the network scan was "normal", a failure due to a "closed port", or a failure due to "no network connection".

Furthermore, when the process proceeds from "YES" in step S1487 to step S1489, the results of the network scan have not been estimated for terminal devices having D scores not smaller than "0" (i.e., D scores greater than or equal to "0" and smaller than (N _terminal ×D_thres)) in step S1489, but after step S1489, steps S1492 to S1497 in FIG. 75 are executed for each terminal device m- _b for which the results of the network scan have not been estimated, so that the results of the network scan for terminal devices having D scores not smaller than "0" (i.e., D scores greater than or equal to "0" and smaller than (N _terminal ×D_thres)) are estimated to be "normal", the cause of the network scan failure is "port closed", or the cause of the network scan failure is "no network connection" (see steps S1494, S1495, S1497).

The steps consisting of "YES" in step S1485 → "NO" in S1491 → "NO" in S1492 → "YES" in S1493 → S1494, and "NO" in step S1485 → S1487 to S1489 correspond to estimating that the network scan is successful or unsuccessful based on the D score D_2 or the D score D_3, and when it is estimated that the network scan has failed, estimating that the network scan has failed due to a cause arising from the communication channel with the terminal device.

In an embodiment of the present invention, the operation of estimating the results of a network scan may be realized by software. In this case, the network scanning device 1 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The ROM stores a program Prog_A consisting of the steps of the flowchart shown in FIG. 66 (including the flowcharts shown in FIGS. 67 to 77, 80 to 83, and the flowchart shown in FIG. 78 or the flowchart shown in FIG. 79).

The CPU reads the program Prog_A from the ROM and executes the read program Prog_A to perform an operation to estimate the results of the network scan. The RAM temporarily stores the various calculation results described above.

The program Prog_A may also be distributed in a form recorded on a recording medium such as a CD or DVD. When the recording medium on which the program Prog_A is recorded is inserted into a computer, the computer reads and executes the program Prog_A from the recording medium, and performs an operation to estimate the results of a network scan.

Therefore, the recording medium on which the program Prog_A is recorded is a computer-readable recording medium.

In the above, it has been described that when classifying terminal devices having communication characteristics CMbest into groups CLS ₁ to CLS ₃ , the estimation means 17 learns an LDA algorithm using scan data when the network scan is “network congested” (communication characteristics CMbest when “network is congested”), and classifies terminal devices having communication characteristics CMbest into three groups CLS ₁ to CLS ₃ using the learned LDA algorithm. However, in the embodiment of the present invention, it is not limited to this, and the estimation means 17 may learn an LDA algorithm using scan data when the most accurately estimated result is obtained from the results of the network scan (= either “normal”, “network congested”, or “inadequate communication quality”), and classify terminal devices having communication characteristics CMbest into three groups CLS ₁ to CLS ₃ using the learned LDA algorithm.

More specifically, when "normal" is the result to be most accurately estimated, the estimation means 17 learns an LDA algorithm using scan data when the network scan is "normal" (communication characteristics CMbest when "normal"), and classifies terminal devices having communication characteristics CMbest into three groups _CLS1 to _CLS3 using the learned LDA algorithm.

Furthermore, when "network congested" is the result to be most accurately estimated, the estimation means 17 learns an LDA algorithm using scan data when the network scan results in "network congested" (communication characteristics CMbest when "network is congested"), and classifies terminal devices having communication characteristics CMbest into three groups _CLS1 to _CLS3 using the learned LDA algorithm.

Furthermore, when "inadequate communication quality" is the result to be most accurately estimated, the estimation means 17 learns an LDA algorithm using scan data when the network scan results in "inadequate communication quality" (communication characteristics CMbest when the communication quality is "inadequate"), and classifies terminal devices having communication characteristics CMbest into three groups _CLS1 to _CLS3 using the learned LDA algorithm.

In this way, the estimation means 17 may determine the scan data that is the teacher data according to the result that is desired to be estimated most accurately, and may learn an LDA algorithm using the determined scan data, and may classify terminal devices having communication characteristics CMbest into three groups _CLS1 to _CLS3 using the learned LDA algorithm.

Also, in the above description, it has been explained that the network scan is performed at each scan timing t _S (or scan timing t' _S ), but this is not limited to the embodiment of the present invention, and the network scan may be performed in units of a scan period T _S _total, or may be performed over a plurality of scan timings within the scan period T _S _total. Therefore, in the embodiment of the present invention, it is sufficient that the communication characteristics acquired by executing the network scan over a plurality of scan timings are used to estimate whether the network scan is "normal" or "failed" by the above-mentioned method, and when it is estimated that the network scan is "failed", the cause of the "failure" of the network scan is further estimated.

Fig. 92 is a diagram showing the probability density function of scan response delay RTT _m . In Fig. 92, the vertical axis represents the probability density function of scan response delay RTT _m , and the horizontal axis represents scan response delay RTT _m . In Fig. 92, (a) represents the probability density function of scan response delay RTT _m acquired in a communication environment where the result of a network scan is "normal", (b) represents the probability density function of scan response delay RTT _m acquired in a communication environment where the result of a network scan is "inadequate communication quality", (c) represents the probability density function of scan response delay RTT _m acquired in a communication environment where the result of a network scan is "network congestion", and (d) represents the probability density function of scan response delay RTT _m acquired in a communication environment where the result of a network scan is "unknown".

Referring to FIG. 92 , the probability density function ((a)) of the scan response delay RTT _m obtained in a communication environment where the result of the network scan is “normal”, the probability density function ((b)) of the scan response delay RTT _m obtained in a communication environment where the result of the network scan is “inadequate communication quality”, and the probability density function ((c)) of the scan response delay RTT _m obtained in a communication environment where the result of the network scan is “network congestion” are all different from each other.

When the probability density function of the scan response delay RTT _m is the probability density function shown in FIG. 92( a), the sample data used to estimate the result of the network scan (= “normal”) is standard sample data D_standard_1. When the probability density function of the scan response delay RTT _m is the probability density function shown in FIG. 92( b), the sample data used to estimate the result of the network scan (= “inadequate communication quality”) is standard sample data D_standard_2. When the probability density function of the scan response delay RTT _m is the probability density function shown in FIG. 92( c), the sample data used to estimate the result of the network scan (= “network congestion”) is standard sample data D_standard_3.

And the probability density function of the scan response delay RTT _m shown in FIG. 92(d) is most similar to the probability density function of the scan response delay RTT _m shown in FIG. 92(a).

Therefore, it has been found that by detecting the probability density function of scan response delay RTT _m most similar to the probability density function of scan response delay RTT m shown in FIG. 92(d) from the three probability density functions of scan response delay RTT _m shown in FIG. 92(a) to (c) in accordance with steps S133 _to S143 shown in FIGS. 69 and 70, it is possible to accurately detect the probability density function of scan response delay RTT _m most similar to the probability density function of scan response delay RTT _m shown in FIG. 92(d).

Furthermore, if the probability density function of scan response delay RTT _m that is most similar to the probability density function of scan response delay RTT _m shown in FIG. 92(d) (= the probability density function of scan response delay RTT _m shown in FIG. 92(a)) is detected, the result of the network scan may be estimated to be “normal”. However, when the result of the network scan is estimated by the estimation algorithm ALG, the result of the network scan is estimated using the communication characteristic CMbest _m in addition to the scan response delay RTT _m , so it cannot be concluded that the estimated result of the network scan will always be “normal”.

Therefore, the estimation algorithm ALG is trained using initial sample data D_sample_initial generated by combining standard sample data D_standard_1 used when the network scan result is estimated to be "normal", standard sample data D_standard_2 used when the network scan result is estimated to be "inadequate communication quality", and standard sample data D_standard_3 used when the network scan result is estimated to be "network congestion", and tentative sample data D_sample_tentative_1 to D_sample_tentative_U generated based on the initial sample data D_sample_initial, and the estimation algorithm ALG is trained using a validation set (=consisting of all of the subsets V _1' to V _U' ) and test data (=subsets TD ₁ to TD _4. ) is used to estimate the results of a network scan using an estimation algorithm ALG (trained estimation algorithm ALG), and the estimation results of the network scan are evaluated using evaluation indexes EVD, nEVD, and tEVD, thereby updating the estimation algorithm ALG so as to approach the optimal algorithm (see the flowcharts shown in FIGS. 69 to 71).

Therefore, by estimating the results of a network scan using the estimation algorithm ALG, which is close to the optimal algorithm, the results of a network scan can be estimated with the highest accuracy.

At the time when the execution of the flowcharts shown in Figures 69 to 71 is completed, the execution of the flowcharts shown in Figures 73 to 75 is completed in step S165 of Figure 71, so the result of the network scan is estimated to be either "normal", "inadequate communication quality", "network congestion", "port closed" or "no connection". If the estimated result of the network scan is "inadequate communication quality" or "network congestion", a rescan list is created in step S15 of Figure 66, and then a network scan is performed based on the rescan list in step S4. As a result, even if the estimated result of the network scan is "inadequate communication quality" or "network congestion", the estimated result of the network scan may become "normal" by rescanning, and the result of the network scan can be accurately estimated.

In an embodiment of the present invention, the estimated result of a network scan consisting of "normal" constitutes the "first estimated result", the estimated result of a network scan consisting of "inadequate communication quality" constitutes the "second estimated result", and the estimated result of a network scan consisting of "network congestion" constitutes the "third estimated result". The first estimated result, the second estimated result, and the third estimated result constitute the "multiple known estimated results".

In addition, in this embodiment of the present invention, the standard sample data D_standard_1 constitutes the "first standard sample data", the standard sample data D_standard_2 constitutes the "second standard sample data", and the standard sample data D_standard_3 constitutes the "third standard sample data".

Furthermore, in an embodiment of the present invention, the first estimation result may be a result estimated by the estimation algorithm ALG using standard sample data D_standard_1, the second estimation result may be a result estimated by the estimation algorithm ALG using standard sample data D_standard_2, and the third estimation result may be a result estimated by the estimation algorithm ALG using standard sample data D_standard_3. Also, each of the first estimation result, the second estimation result, and the third estimation result may be a result estimated by an algorithm other than the estimation algorithm ALG.

In the embodiment of the present invention, the threshold value _{Corr_r} constitutes a "first threshold value", the threshold value _Sr constitutes a "second threshold value", and the threshold value (N _terminal ×D_thres) constitutes a "third threshold value".

In addition, in this embodiment of the present invention, the communication characteristic CMbest constitutes a "first communication characteristic", the communication characteristics having a correlation coefficient r _ij greater than the threshold value Corr _r constitute "D (D is a positive integer) second communication characteristics", and the communication characteristics extracted in descending order of the score Q constitute "K (K is a positive integer) second communication characteristics".

Furthermore, in an embodiment of the present invention, the D score or count value DS_count constitutes a "discrimination score."

Furthermore, in this embodiment of the invention, the D score D_2 constitutes the "first discrimination score", the D score D_3 constitutes the "second discrimination score", and the D score D_1 constitutes the "third discrimination score".

Furthermore, in this embodiment of the present invention, the packet loss rate PLR _m , jitter JT _m , scanning maximum delay DLmax _m , and scanning minimum delay DLmin _m shown in FIG. 12 constitute "D (D is a positive integer) second communication characteristics" whose correlation coefficient r _ij is greater than a threshold value Corr _r .

Furthermore, in an embodiment of the present invention, the K communication characteristics described above constitute "K second communication characteristics."

Furthermore, in this embodiment of the present invention, group CLS ₁ constitutes a "first group" in which the network scan is normal, and groups CLS ₂ and CLS ₃ constitute a "second group" and a "third group" respectively in which the network scan has failed due to a cause attributable to the communication channel with the terminal device.

Furthermore, in this embodiment of the invention, Model 2 constitutes the "first model" and Model 3 constitutes the "second model."

The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is indicated by the claims, not by the description of the embodiments above, and is intended to include all modifications within the meaning and scope of the claims.

This invention applies to a network scanning device, a program for causing a computer to execute the program, a computer-readable recording medium on which the program is recorded, and a data production method.

1 Network scanning device, 10 Communication system, 11 Network scanner, 12 Scan analysis manager, 13 Host information database, 14 Update means, 15 Extraction means, 16 Calculation means, 17 Estimation means, 18 Scan scheduler, 20 Scan target network, 30 User.

Claims (22)

a scanning means for executing a network scan in a first communication environment in which the results of a network scan are unknown for M (M is an integer equal to or greater than 2) terminal devices, and acquiring a first scan response delay of the network scan for each of the M terminal devices;
a calculation means for executing a first calculation process, when the M first scan response delays are acquired by the scanning means, for calculating the M' baseline scan response delays for each of a plurality of sets of M'(M' is an integer of 2 or more) second scan response delays obtained when results of a network scan executed in a second communication environment in which results of a network scan are known are a plurality of known estimated results, and for calculating the M baseline scan response delays for the M first scan response delays; and a second calculation process, based on the plurality of sets of the M' baseline scan response delays and the M baseline scan response delays calculated in the first calculation process, for calculating similarity rates between the M' baseline scan response delays and the M baseline scan response delays for all of the plurality of sets of M' baseline scan response delays to calculate a plurality of similarity rates.
a detection means for detecting a largest similarity rate among the plurality of similarity rates calculated in the second calculation process when the second calculation process is executed by the calculation means;
a determination means for determining, when the highest similarity rate is detected by the detection means, that an unknown estimation result of a network scan when the M first scan response delays are acquired is the known estimation result corresponding to the M′ second scan response delays having the highest similarity rate;
The baseline scan response delay is a scan response delay whose distribution is changed based on an "upper limit" and a "lower limit" so as to more easily obtain a similarity or difference in distribution between a scan response delay obtained when a network scan is performed in a communication environment where the results of the network scan are unknown and a scan response delay obtained when a network scan is performed in a communication environment where the results of the network scan are known. The network scanning device according to claim 1, further comprising an update means for executing an aggregation process that aggregates the M' second scan response delays and the M first scan response delays when the highest similarity rate is obtained, and updating the M' second scan response delays when the highest similarity rate is obtained by the M' second aggregated scan response delays aggregated in the aggregation process. The network scanning device according to claim 2, wherein, when there are multiple sets of the M first scan response delays when the highest similarity rate is obtained, the update means selects an arbitrary set of M first scan response delays from the multiple sets of M first scan response delays in the aggregation process, and aggregates the selected set of M first scan response delays with M' second scan response delays when the highest similarity rate is obtained. The network scanning device according to claim 2 or 3, wherein the update means, in the aggregation process, adds the first scan response delays, among the M first scan response delays, that are not included in the M' second scan response delays when the highest similarity rate is obtained, to the M' second scan response delays when the highest similarity rate is obtained, thereby aggregating the M' second scan response delays when the highest similarity rate is obtained and the M first scan response delays. the calculation means, in the first calculation process, executes a process of calculating, as the baseline scan response delay, a subtraction result obtained by subtracting a predetermined value from an average of a plurality of scan response delays and an addition result obtained by adding the predetermined value to the average, for all of the plurality of scan response delays to calculate a plurality of baseline scan response delays, the calculation means executes a process of calculating, for all of the M first scan response delays and the plurality of sets of M' second scan response delays, to calculate the plurality of sets of M' baseline scan response delays and the M baseline scan response delays;
5. The network scanning device according to claim 1, wherein the predetermined value is calculated based on relative variations in the plurality of scan response delays. The network scanning device according to claim 5, wherein in the first calculation process, the calculation means subtracts the inverse of the coefficient of variation in the distribution of the plurality of scan response delays from the scan response delay, multiplies the result by the standard deviation of the plurality of scan response delays, and divides the result by the number of the plurality of scan response delays to calculate the predetermined value. 7. The network scanning device according to claim 1, wherein in the second calculation process, the calculation means calculates a first mean absolute deviation which is a mean absolute deviation of the M baseline scan response delays, calculates a second mean absolute deviation which is a mean absolute deviation of the M' baseline scan response delays for all of the multiple sets of M' baseline scan response delays to calculate a plurality of second mean absolute deviations, and calculates the similarity rates for all of the multiple second mean absolute deviations by a formula of 1/(1+(Y-X) ² /XY), where X is the first mean absolute deviation and Y is any of the multiple second mean absolute deviations. a generating means for acquiring, for each of the M' second scan response delays, an optimal communication characteristic that is the communication characteristic most suitable for estimating the known estimation result among a plurality of communication characteristics acquired together with one of the M' second scan response delays, and generating similar standard sample data consisting of the acquired M' optimal communication characteristics and the M' second aggregate scan response delays;
an algorithm updating means for training an estimation algorithm for estimating the result of the network scan executed in the first communication environment based on the similar standard sample data, and updating the estimation algorithm using an evaluation index so as to approach an optimal algorithm;
an estimation means for estimating a result of a network scan executed in the first communication environment by using the estimation algorithm updated by the algorithm update means,
the optimization algorithm comprises an algorithm for estimating a result of a network scan such that the estimated result of the network scan matches the known estimated result;
8. The network scanning device according to claim 1, wherein the evaluation index is an index for evaluating accuracy of an estimation result when a result of a network scan is estimated by the trained algorithm. A first step in which a scanning means executes a network scan in a first communication environment in which the results of a network scan are unknown for M (M is an integer equal to or greater than 2) terminal devices, and acquires a first scan response delay of the network scan for each of the M terminal devices;
a second step in which a calculation means, when the M first scan response delays are acquired by the scanning means, executes a first calculation process in which the calculation means calculates the M' baseline scan response delays for each of a plurality of sets of M'(M' is an integer of 2 or more) second scan response delays obtained when results of a network scan executed in a second communication environment in which results of a network scan are known are a plurality of known estimated results, and calculates the M baseline scan response delays for the M first scan response delays; and a second calculation process in which the calculation means calculates similarity rates between the M' baseline scan response delays and the M baseline scan response delays for all of the plurality of sets of M' baseline scan response delays based on the plurality of sets of the M' baseline scan response delays and the M baseline scan response delays calculated in the first calculation process, to calculate a plurality of similarity rates.
a third step of detecting a largest similarity rate among the plurality of similarity rates calculated in the second calculation process by the detection means when the second calculation process is executed by the calculation means;
a fourth step in which a determination means determines, when the highest similarity rate is detected in the third step, that an unknown estimation result of a network scan when the M first scan response delays are acquired is the known estimation result corresponding to the M′ second scan response delays having the highest similarity rate;
The baseline scan response delay is a scan response delay whose distribution has been changed based on an "upper limit" and a "lower limit" so as to more easily produce a similar or different distribution between a scan response delay obtained when a network scan is performed in a communication environment where the results of the network scan are unknown and a scan response delay obtained when a network scan is performed in a communication environment where the results of the network scan are known. The program for causing a computer to execute the program according to claim 9, further comprising a fifth step in which, when the highest similarity rate is detected in the third step, the update means executes an aggregation process for aggregating the M first scan response delays and the M' second scan response delays when the highest similarity rate is obtained, and updates the M' second scan response delays when the highest similarity rate is obtained by the M' second aggregated scan response delays aggregated in the aggregation process. The program for causing a computer to execute the program according to claim 10, wherein, when there are multiple sets of the M first scan response delays when the highest similarity rate is obtained, the update means selects an arbitrary set of M first scan response delays from the multiple sets of M first scan response delays in the aggregation process, and aggregates the selected set of M first scan response delays with M' second scan response delays when the highest similarity rate is obtained. The program for causing a computer to execute the program according to claim 10 or 11, wherein the updating means, in the aggregation process, adds the first scan response delays among the M first scan response delays that are not included in the M' second scan response delays when the highest similarity rate is obtained to the M' second scan response delays when the highest similarity rate is obtained, thereby aggregating the M' second scan response delays when the highest similarity rate is obtained and the M first scan response delays. the calculation means, in the first calculation process, executes a process of calculating a baseline scan response delay by subtracting a predetermined value from an average of a plurality of scan response delays and an addition result by adding the predetermined value to the average for all of the plurality of scan response delays to calculate a plurality of baseline scan response delays, the calculation means ... plurality of baseline scan response delays by subtracting a predetermined value from an average of a plurality of scan response delays and an addition result by adding the predetermined value to the average for all of the plurality of scan response delays to calculate the plurality of sets of M' second scan response delays,
13. The program executed by a computer according to claim 9, wherein the predetermined value is calculated based on relative variations of the plurality of scan response delays. The program for causing a computer to execute the program according to claim 13, wherein the calculation means, in the first calculation process, multiplies a subtraction result obtained by subtracting an inverse of a coefficient of variation in the distribution of the plurality of scan response delays from the scan response delay by a standard deviation of the plurality of scan response delays, and divides the multiplication result by the number of the plurality of scan response delays to calculate the predetermined value. 15. The program for causing a computer to execute the program according to claim 9, wherein, in the second calculation process, the calculation means calculates a first mean absolute deviation that is a mean absolute deviation of the M baseline scan response delays, calculates a second mean absolute deviation that is a mean absolute deviation of the M' baseline scan response delays for all of the multiple sets of M' baseline scan response delays to calculate multiple second mean absolute deviations, and calculates the similarity rates for all of the multiple second mean absolute deviations by an equation of 1/(1+(Y-X) ² /XY), where X is the first mean absolute deviation and Y is any of the multiple second mean absolute deviations. a sixth step in which a generating means acquires, for each of the M' second scan response delays, an optimal communication characteristic that is the communication characteristic most suitable for estimating the known estimation result among a plurality of communication characteristics acquired together with one of the M' second scan response delays, and generates similar standard sample data consisting of the acquired M' optimal communication characteristics and the M' second aggregate scan response delays;
a seventh step in which an algorithm update means trains an estimation algorithm for estimating a result of a network scan executed in the first communication environment based on the similar standard sample data, and updates the estimation algorithm using an evaluation index so as to approach an optimal algorithm;
and an eighth step in which an estimation means estimates a result of a network scan performed in the first communication environment using the estimation algorithm updated by the algorithm update means,
the optimization algorithm comprises an algorithm for estimating a result of a network scan such that the estimated result of the network scan matches the known estimated result;
16. The program for causing a computer to execute the program according to claim 9, wherein the evaluation index is an index for evaluating the accuracy of an estimation result when a result of a network scan is estimated by the trained algorithm. A computer-readable recording medium having recorded thereon the program according to any one of claims 9 to 16. 1. A data production method for producing data used to estimate results of a network scan, comprising:
A first step in which a scanning means executes a network scan in a first communication environment in which the results of a network scan are unknown for M (M is an integer equal to or greater than 2) terminal devices, and acquires a first scan response delay of the network scan for each of the M terminal devices;
a second step in which a calculation means executes a first calculation process, when the M first scan response delays are acquired in the first step, to calculate the M' baseline scan response delays for each of a plurality of sets of M'(M' is an integer of 2 or more) second scan response delays obtained when results of a network scan executed in a second communication environment in which results of a network scan are known are a plurality of known estimated results, and to calculate the M baseline scan response delays for the M first scan response delays; and a second calculation process, based on the plurality of sets of the M' baseline scan response delays and the M baseline scan response delays calculated in the first calculation process, to calculate similarity rates between the M' baseline scan response delays and the M baseline scan response delays for all of the plurality of sets of M' baseline scan response delays to calculate a plurality of similarity rates;
a third step of detecting a largest similarity rate among the plurality of similarity rates calculated in the second calculation process by a detection means when the second calculation process is executed in the second step;
a fourth step in which an updating means executes an aggregation process of aggregating the M' second scan response delays and the M first scan response delays when the highest similarity rate is obtained, when the highest similarity rate is detected in the third step, and updates the M' second scan response delays when the highest similarity rate is obtained by the M' second aggregated scan response delays aggregated in the aggregation process;
a fifth step in which a generating means acquires, for each of the M' second scan response delays, an optimal communication characteristic that is the communication characteristic most suitable for estimating the known estimation result among the multiple communication characteristics acquired together with one of the M' second scan response delays updated in the fourth step, and generates similar standard sample data consisting of the acquired M' optimal communication characteristics and the M' second scan response delays;
A data production method in which the baseline scan response delay is a scan response delay whose distribution is changed based on an "upper limit" and a "lower limit" so as to more easily produce similar or different distributions between a scan response delay obtained when a network scan is performed in a communication environment where the results of the network scan are unknown and a scan response delay obtained when a network scan is performed in a communication environment where the results of the network scan are known. 19. The data manufacturing method of claim 18 , wherein the updating means, in the aggregation process of the fourth step, adds the first scan response delays among the M first scan response delays that are not included in the M' second scan response delays when the highest similarity rate is obtained to the M' second scan response delays when the highest similarity rate is obtained, thereby aggregating the M' second scan response delays when the highest similarity rate is obtained and the M first scan response delays. the calculation means, in the first calculation process of the second step, executes a process of calculating a baseline scan response delay by subtracting a predetermined value from an average of a plurality of scan response delays and an addition result by adding the predetermined value to the average for all of the plurality of scan response delays to calculate a plurality of baseline scan response delays, the calculation means executes a process of calculating a baseline scan response delay by subtracting a predetermined value from an average of a plurality of scan response delays and an addition result by adding the predetermined value to the average for all of the M first scan response delays and the plurality of sets of M' second scan response delays to calculate the M' baseline scan response delays and the plurality of sets of M baseline scan response delays,
20. The data manufacturing method according to claim 18, wherein the predetermined value is calculated based on relative variations of the plurality of scan response delays. 21. The data manufacturing method according to claim 20, wherein the calculation means, in the first calculation process of the second step, multiplies a subtraction result obtained by subtracting an inverse of a coefficient of variation of the plurality of scan response delays from the scan response delay by a standard deviation of the plurality of scan response delays, and divides the multiplication result by the number of the plurality of scan response delays to calculate the predetermined value. 22. The data manufacturing method according to claim 18, wherein in the second calculation process of the second step, the calculation means calculates a first mean absolute deviation which is a mean absolute deviation of the M baseline scan response delays, calculates a second mean absolute deviation which is a mean absolute deviation of the M' baseline scan response delays for all of the multiple sets of M' baseline scan response delays to calculate a plurality of second mean absolute deviations, and calculates the similarity rates by a formula of 1/(1+(Y-X) ² /XY), where X is the first mean absolute deviation and Y is any one of the multiple second mean absolute deviations, for all of the multiple second mean absolute deviations to calculate the multiple similarity rates.

Images