JP6520199B2 - Extraction method, information processing apparatus, and extraction program - Google Patents

Description

  The present invention relates to an extraction method, an information processing apparatus, and an extraction program.

  Social simulation is to model human behavior and interaction, and simulate social systems with uncertainty on a computer. In the social simulation, the trial is repeated while changing the characteristics of the element having uncertainty with respect to the measure by random numbers and the like for each of the plurality of measures for the subject of the simulation.

  For example, if the theme is to reduce the wait time of the cash register of the store, the number and layout of the cash register of the store will be the measure, and the behavior characteristic of the customer as the characteristic of the element having uncertainty with respect to the measure It is conceivable to carry out a simulation (see, for example, Patent Document 1). In this case, the trial is repeated while changing the behavior characteristic of the customer for each of a plurality of measures regarding the number and layout of the cash register. As an example of the behavior characteristic of the customer, the arrival time of the customer, the number of purchases of goods, and the like can be mentioned.

  In social simulation, problems and the like of measures can be extracted by analyzing the results of a plurality of trials. The greater the number of trials, the more cases can be considered and the more problems are likely to be extracted with greater precision.

JP, 2013-250941, A JP 2004-334328 A JP, 2009-225550, A JP 2005-157788 A

  However, it is not economical to analyze each of a large number of trials. For example, in the simulation regarding the number and layout of the cash register of the store, when considering the behavior characteristic of the customer of 100 patterns for the number and layout of 100 patterns, 10000 trials will be performed.

  Therefore, in one aspect, the present invention aims to extract an analysis target trial from a plurality of simulation trials.

  In one scheme, the extraction method is obtained by performing the simulation trial on a plurality of characteristics of an element having uncertainty with respect to the policy for each of the plurality of policies on the subject of the simulation. The process of acquiring the output value of each of the plurality of trials for each measure and the characteristic, and the output according to the first trial group among the plurality of trials, for each first trial group having the same characteristic. The plurality of trials based on the process of calculating the difference between the value group and the average value of each of the plurality of trials and the difference between the plurality of trials. The computer executes a process of extracting some trials from

  According to one aspect, an analysis target trial can be extracted from a plurality of simulation trials.

It is a figure for demonstrating the outline | summary of 1st embodiment. It is a figure for demonstrating an example of the extraction method of a characteristic trial. It is a figure for demonstrating normalization of an evaluation value. It is a figure showing the example of hardware constitutions of the extraction device in a first embodiment. It is a figure which shows the function structural example of the extraction apparatus in 1st embodiment. It is a flowchart for demonstrating an example of the process sequence which the extraction apparatus of 1st embodiment performs. It is a figure which shows an example of the simulation result of 1st embodiment. It is a figure which shows an example of a normalization value. It is a figure which shows an example of the calculation result of the difference with the average value for every environmental characteristic of each normalization value. It is a figure showing an example of extraction of a value whose absolute value exceeds a threshold. It is a figure for demonstrating the outline | summary of 2nd embodiment. It is a figure which shows the function structural example of the extraction apparatus in 2nd embodiment. It is a flowchart for demonstrating an example of the process sequence of the classification process of the trial in 2nd embodiment. It is a figure which shows an example of the simulation result of 2nd embodiment. It is a figure which shows an example of the classification result of each trial based on the evaluation value of each parameter | index. It is a flowchart for demonstrating an example of the process sequence of the extraction process of the trial in 2nd embodiment. It is a flowchart for demonstrating an example of the process sequence of the update process of an extraction candidate. It is a figure which shows the function structural example of the extraction apparatus in 3rd embodiment.

  Hereinafter, a first embodiment will be described based on the drawings. FIG. 1 is a diagram for explaining the outline of the first embodiment. A graph g1 shown in FIG. 1 shows an execution result of a certain simulation. In the present embodiment, the behavior of the customer is modeled, and a simulation is repeated in which trials are repeated for behavior characteristics of a plurality of customers for each of the number and layout of a plurality of cash registers. The subject of the simulation is to reduce the waiting time of the cash register of the store. The number and layout of cash register are the measures for the subject. The subject is, for example, a subject to be considered for a simulation performer or a requester. The measure is, for example, a measure considered for the subject. In the simulation, the measure is a matter that can be changed at the intention of the measure planner. On the other hand, it is difficult for the policy planner's intention to control elements that have uncertainty with the policy, such as customer behavior. Hereinafter, the characteristic of such an element is hereinafter referred to as "environmental characteristic".

  Here, the environmental characteristic is not the behavior (movement) of the customer as an example of the virtual entity in the simulation but a restriction given to the customer's behavior in the simulation, or a distribution regarding the customer's behavior. For example, the number of customers per hour, the average and the standard deviation of the number of purchases of products by customers per person, and the like are examples of environmental characteristics. In the simulation, random numbers are used, for example, based on the average number and standard deviation of the number of customers per hour and the purchase points of the customers per person, and the timing of appearance of each customer and each The purchase points of the customers are determined. Therefore, in a trial in which environmental characteristics are common to each other, the behavior of each customer does not always match completely. In addition, trial means execution of one simulation performed with respect to one measure and one environmental characteristic.

  In the graph g1, the horizontal axis is a measure, and the vertical axis is an evaluation value output for each trial. The graph g1 indicates that trials have been performed on nine types of measures 1 to 9. An evaluation value is an output value of a trial, and is used, for example, for evaluation of a measure. In the present embodiment, the maximum value (maximum waiting time) of the waiting times of each customer appearing in one trial is taken as the evaluation value. The waiting time is the time required for the modeled customer to reach the end of the cash register queue and to leave the cash register.

  In the graph g1, a plot vertically arranged for each measure is an evaluation value of each trial related to the measure. FIG. 1 shows an example in which 42 trials were performed for each measure. That is, an example is shown in which trials were performed for 42 environmental characteristics for each of the 9 types of measures. Therefore, the total number of trials is 9 × 42 = 378 times.

  In the graph g1, the broken line is the connection of the average value of the evaluation value group for each measure. On the other hand, the plots shown by triangles (Δ) are plots for certain environmental characteristics. That is, the group of plots is a set of plots in which environmental characteristics are common and measures are mutually different.

  Among the plot groups, the plots excluding the plots for measures 2 and 3 show almost the same evaluation values. On the other hand, measures 2 and 3 are far from the other plot groups.

  Therefore, under the same environmental characteristics (for example, in the situation where the behavior of customers is similar), measures 2 and 3 have a risk that congestion may occur around the cash register that does not occur with other measures I understand. Then, it can be said that it is effective to analyze the trial concerning the combination of the environmental characteristic and the measure 2 or the measure 3 to avoid such a risk.

  Therefore, in the present embodiment, it is considered that the trial corresponding to the plot of Δ in the measure 2 or 3 is a try that can indicate a problem specific to the measure. That is, in the present embodiment, a method of extracting a trial related to an evaluation value which deviates from other evaluation values in a trial group having a large variation in evaluation value depending on the measure despite common environmental characteristics. Is disclosed. Hereinafter, such a trial is simply referred to as a “characteristic trial”.

  In addition, if the trial which shows simple statistics values, such as the maximum value or the minimum value, is extracted, it is difficult to extract the problem point of each measure. That is, the trial showing the maximum value or the minimum value is highly likely to be largely dependent on the characteristics of the element having uncertainty, such as a large number of customers having a large number of purchases concentrated in a short period of time. Therefore, even if the maximum value or the minimum value is analyzed, there is a high possibility that common problems (for example, concentration of customers) are extracted in any of the measures, and problems specific to certain measures There is a high possibility that it can not be extracted.

  In the present embodiment, characteristic trials are extracted by, for example, the method shown in FIG. FIG. 2 is a diagram for explaining an example of a characteristic trial extraction method. In FIG. 2, the same parts as in FIG. 1 are given the same reference numerals.

  For example, in a set of evaluation values as shown in graph g1, for each evaluation value group (hereinafter referred to as "common environmental evaluation value group") related to a trial group having common environmental characteristics, the environmental common evaluation value group An average value is calculated, and for each evaluation value belonging to the environmental common evaluation value group, a difference from the average value is calculated (S1). Since the graph g1 is a trial result regarding 42 environmental characteristics, an average value and a difference are calculated for each of 42 environmental common evaluation value groups.

  The graph g2 is a plot of the difference. The horizontal axis in the graph g2 is a measure similar to the graph g1. The vertical axis is the difference from the average value for each evaluation value for each environmental common characteristic group.

  Subsequently, characteristic trials are extracted based on the difference from the average value. For example, a difference larger than the positive threshold and a difference smaller than the negative threshold are extracted (S2). The graph g3 is a combination of a graph indicating the range a1 larger than the positive threshold and a graph indicating the range a2 smaller than the negative threshold with respect to the graph g2. In the graph g3, the trial concerning the plot included in the range a1 or the range a2 is extracted as a characteristic trial.

  In addition, as a pre-processing before the extraction method shown in FIG. 2 is performed, the average value and the dispersion (standard deviation) of each evaluation value group for each measure (hereinafter referred to as "measure common evaluation value group") Normalization may be performed on each evaluation value so as to be mutually consistent among the measure common evaluation value group.

  FIG. 3 is a diagram for explaining normalization of the evaluation value. In FIG. 3, the same parts as in FIG. 1 are given the same reference numerals. In FIG. 3, a graph g4 is a result of normalizing each evaluation value such that the average value of each measure common evaluation value group is 0 and the standard deviation is 1.

  The extraction method described in FIG. 2 may be performed on the normalized set of evaluation values. By doing so, when there is a measure that the average value and variance of the measure common evaluation value group differ greatly from other measures, the possibility that many trials related to the measure will be extracted as characteristic trials is reduced. be able to. That is, the average impact of each measure can be removed, and the possibility of extracting characteristic trials can be enhanced in each measure.

  For example, suppose that there is a measure in which the average value of the measure common evaluation value group is extremely small compared to other measures. The maximum value of the measure common evaluation value group of the measure is an extremely large value in the measure common evaluation value group, but the trial concerning another measure implemented for the same environmental characteristic as the environmental characteristic concerning the maximum value It is assumed that the evaluation value group is an approximate value. In such a case, since the difference from the average value calculated with respect to the maximum value is close to 0, the extraction method illustrated in FIG. 2 may not extract a trial related to the maximum evaluation value.

  However, the trial suggests that there is a risk that the maximum waiting time is extremely long in certain environmental characteristics, when viewed on the basis of the average value of the measure common evaluation value group of the measure. Therefore, by analyzing the trial, there is a possibility that the cause of the risk can be elucidated. Therefore, it can be said that the trial is also a characteristic trial.

  Therefore, the above normalization may be performed so that characteristic trials that may be buried by the average tendency of the measure can be extracted.

  Hereinafter, details of the extraction apparatus 10 for realizing the trial extraction method described above will be described.

  FIG. 4 is a diagram showing an example of a hardware configuration of the extraction device in the first embodiment. The extraction device 10 of FIG. 4 includes a drive device 100, an auxiliary storage device 102, a memory device 103, a CPU 104, an interface device 105, and the like mutually connected by a bus B.

  A program for realizing the processing in the extraction device 10 is provided by the recording medium 101. When the recording medium 101 in which the program is recorded is set in the drive apparatus 100, the program is installed from the recording medium 101 to the auxiliary storage apparatus 102 via the drive apparatus 100. However, the installation of the program does not necessarily have to be performed from the recording medium 101, and may be downloaded from another computer via a network. The auxiliary storage device 102 stores the installed program and also stores necessary files and data.

  The memory device 103 reads out the program from the auxiliary storage device 102 and stores it when there is an instruction to start the program. The CPU 104 executes a function related to the extraction device 10 in accordance with a program stored in the memory device 103. The interface device 105 is used as an interface for connecting to a network.

  Note that, as an example of the recording medium 101, a portable recording medium such as a CD-ROM, a DVD disk, or a USB memory can be mentioned. Further, as an example of the auxiliary storage device 102, a hard disk drive (HDD) or a flash memory may be mentioned. The recording medium 101 and the auxiliary storage device 102 both correspond to computer readable recording media.

  FIG. 5 is a diagram showing an example of a functional configuration of the extraction device in the first embodiment. In FIG. 5, the extraction apparatus 10 includes a result acquisition unit 11, a normalization unit 12, a difference calculation unit 13, an extraction unit 14, an output unit 15, and the like. These units are realized by processing that one or more programs installed in the extraction device 10 cause the CPU 104 to execute. The extraction device 10 also uses the simulation result storage unit 16. The simulation result storage unit 16 can be realized using the auxiliary storage device 102 or a storage device connected to the extraction device 10 via a network.

  The simulation result storage unit 16 stores simulation results. That is, the simulation result storage unit 16 stores the evaluation value output by the trial for each trial of the simulation. The result acquisition unit 11 acquires the simulation result from the simulation result storage unit 16.

  The normalization unit 12 performs normalization as described in FIG. 3 on each evaluation value included in the simulation result acquired by the result acquisition unit 11. The difference calculating unit 13 calculates an average value for each environmental common evaluation value group for the normalized set of evaluation values, and calculates a difference between the common environmental evaluation value group and the average value of each evaluation value. The extraction unit 14 extracts characteristic trials based on the difference calculated for each evaluation value by the difference calculation unit 13. The output unit 15 outputs information related to the trial extracted by the extraction unit 14 (for example, information indicating a measure and information indicating an environmental characteristic).

  Hereinafter, the processing procedure which extraction device 10 performs is explained. FIG. 6 is a flowchart for explaining an example of the processing procedure executed by the extraction device of the first embodiment.

  In step S101, the result acquisition unit 11 acquires a simulation result from the simulation result storage unit 16.

FIG. 7 is a view showing an example of a simulation result of the first embodiment. In FIG. 7, the evaluation value output by the trial which concerns on the said combination is shown for every combination with each of three types of measures of the measures 1-3, and each of five kinds of environmental characteristics of the environmental characteristics 1-5. It is shown. Hereinafter, the i-th measure is referred to as "measure i", and the j-th environmental characteristic is referred to as "environmental characteristic j". Further, the evaluation value of the trial concerning the combination of the measure i and the environmental characteristic _j is denoted as _{Ei, j} .

Subsequently, the normalization unit 12 normalizes each E _{i, j} so that the average value and the variance of each measure common evaluation value group coincide with each other (S102). Specifically, the normalization unit 12 calculates an average value and a variance for each measure common evaluation value group (each row in FIG. 7). For the evaluation value _{Ei, j} belonging to the measure common evaluation value group pertaining to the i-th measure, the normalization unit 12 applies the mean value and the variance of the measure common evaluation value group to the following formula (1) , A normalized evaluation value (hereinafter referred to as “normalized value”) F _{i, j} is obtained.
F _{i, j} = (average value of E _{i, j −} i-th measure common evaluation value group) / (dispersion of i-th measure common evaluation value group) ^1/2 (1)
For example, the average value of the measure common evaluation value group according to measure 1 is 13.8, and the variance is 11.7. Therefore, for the normalization unit 12, j = 1 to 5
F _{1, j} = (E _{1, j-} 13.8) / (11.7) ^1/2
Calculate Similarly, all F _{i, j} are calculated.

FIG. 8 is a diagram showing an example of the normalized value. According to equation (1), _{F i} for each _measure, next average value of _j is 0, _{F i} for each _measure, as the standard deviation of the _j becomes 1, each _{E i,} for _{j F i, j} is It is calculated.

Subsequently, the difference calculating unit 13 calculates an average value for each F _{i, j} (for each row in FIG. 8) in which the environmental characteristics are common (S103). For example, for environmental characteristic 1, the average value of F _{i, 1} for i = 1 to 3 is 0.266928.

Subsequently, the difference calculating unit 13 calculates _, for each F _{i, j} common to the environmental characteristics, the difference from the average value calculated for the environmental characteristics (S104). For example, assuming that the difference between F _{i, j} and the average value calculated for the j-th environmental characteristic is G _{i, j} , calculation is performed based on the following equation (2).
G _{i, j} = average value calculated for the environmental characteristic of the f _{i, j} −th (2)
As described above, the average value calculated for environmental characteristic 1 is 0.266928. Therefore, for measures i = 1 to 3,
Gi _{, 1} = Fi _, 1-0.266928
Is calculated. Similarly, all G _{i, j} are calculated.

  FIG. 9 is a diagram showing an example of the calculation result of the difference between each normalized value and the average value for each environment characteristic.

  Subsequently, the extraction unit 14 extracts values larger than the positive threshold or smaller than the negative threshold among the values shown in FIG. 9 (S105). That is, among the values shown in FIG. 9, values whose absolute values exceed the threshold are extracted. However, the absolute values of the positive threshold and the negative threshold may not be the same. Each threshold value is stored in advance in the auxiliary storage device 102, for example.

FIG. 10 is a diagram illustrating an example of extraction of a value whose absolute value exceeds a threshold. In FIG. 10, values surrounded by thick lines are extracted values. Specifically, the measures 2 and trial _2,1 environmental characteristics 1, and measures 1 and trial ₂ environment characteristic 2, and trial ₂ measures 2 and environmental properties 3 is extracted. In other words, the trial concerning the value is extracted. That is, FIG. 10 shows the extraction result in the case where the positive threshold is 1.0 and the negative threshold is -1.0. Each of these threshold values is a value that is separated (out of the standard deviation) by 1 from the average value for each environmental characteristic. The absolute value of the threshold is not limited to one.

  Subsequently, the output unit 15 outputs information on the trial extracted by the extraction unit 14 (S106). For example, information indicating a measure of the trial and information indicating an environmental characteristic may be output. Also, the output form is not limited to a predetermined one. For example, the information may be displayed on a display device or may be transmitted to a computer connected via a network. In addition, the information may be stored in the auxiliary storage device 102 or the like, or may be output to a printer.

For example, when the top three evaluation values are to be extracted, in FIG. 7, trials _{2, 1} and ₁ , ₄ with an evaluation value of 18 and trials ₃ and ₄ with an evaluation value of 16 are extracted. Be done. On the other hand, the trials extracted in this embodiment are trials ₂ , ₁ , trials ₁ , ₂ and trials ₂ , ₃ . Among these, trials ₁ and ₂ and trials ₂ and ₃ are not included in the top third of _the evaluation value. Therefore, according to the present embodiment, it is possible to extract trials relating to evaluation values that do not correspond to simple statistical values. As a result, it is possible to extract trials that may indicate problems specific to the policy.

  In the present embodiment, the case where the number of trials is small has been described, but even if a large number of trials have been performed, for example, 10000, the characteristic trial can be simplified if the extraction method according to the present embodiment is used. Can be extracted.

  In FIG. 6, step S102 may be omitted. In this case, in step S103, the simulation result shown in FIG. 7 may be used as input data.

  Moreover, the range which can apply this Embodiment is not limited to the result of a specific simulation. The present embodiment is applicable to various simulation results as long as the simulation is repeated for each of a plurality of environmental characteristics for each of a plurality of measures.

  For example, it is assumed that there is an organization including a plurality of call centers each having different business content. In this organization, a job rotation is provided for each operator, and each operator moves a call center at regular intervals and takes charge of different tasks. Here, if the rotation interval (the duration of one operation) is too long, each operator may forget the contents of the original operation when the rotation makes one round, and the operation efficiency may be reduced. On the other hand, if the interval is too short, each operator will be in charge of the next task without becoming familiar with the task in charge, and there is a possibility that work efficiency may be reduced. Therefore, it is assumed that it is required to obtain an optimal value for the interval.

  In this case, it is conceivable that the simulation is executed with the interval being a measure and the number of calls, the timing of calls, and the like being environmental characteristics. That is, it is conceivable that the trial is repeated for a plurality of environmental characteristics for each of the plurality of intervals. From each trial, for example, the work efficiency may be output as an evaluation value.

  It is apparent that the present embodiment is effective also for the result of simulation performed in this manner.

  In addition, although the extraction apparatus 10 concerning this embodiment extracts a characteristic trial, based on the extracted characteristic trial, you may perform a further simulation. That is, the extraction device 10 may perform a detailed analysis of the extracted characteristic trials. In this case, the extraction apparatus 10 may receive, as an input, a combination of environments and characteristics corresponding to the extracted characteristic trials, and may execute a simulation for detailed analysis.

  Also, instead of the difference from the average value, a statistical value such as a deviation value may be used.

  Next, a second embodiment will be described. In the second embodiment, points different from the first embodiment will be described. The second embodiment may be the same as the first embodiment in the points not particularly mentioned.

  In the second embodiment, a case where there are a plurality of indices used as evaluation values for each trial will be described. For example, in the first embodiment, the maximum value of the waiting time of the cash register is taken as an example of the index of the evaluation value, but in the second embodiment, for example, in addition to the maximum value of the waiting time of the cash register An example is described in which the average value of the waiting time is used as an index of the evaluation value.

  For example, with respect to 100 measures and 100 environmental characteristics, 10000 trials are performed as in the first embodiment. However, in the second embodiment, two types of evaluation values are output for each trial. Therefore, it is necessary to extract trials to be analyzed (refine trials) based on 10000 × 2 = 20000 evaluation values.

  FIG. 11 is a diagram for explaining an outline of the second embodiment. FIG. 11 shows evaluation values output by eight trials performed on four environmental characteristics for each of two types of measures. (1) shows an evaluation value concerning index 1, and (2) shows an evaluation value concerning index 2. That is, FIG. 11 shows an example in which evaluation values regarding two types of indexes of index 1 and index 2 are output by one trial.

  In the present embodiment, first, for each index, the trial group is classified into a plurality of groups based on the comparison between the evaluation value of the index and the two threshold values of the index. Two thresholds are an upper threshold and a lower threshold. The upper threshold is a threshold for extracting or identifying the trial in which the evaluation value is greater. The lower threshold is a threshold for extracting or identifying an attempt with a smaller evaluation value. A group whose evaluation value exceeds the upper threshold is called "upper classification". A group whose evaluation value is less than or equal to the upper threshold and greater than or equal to the lower threshold is referred to as "middle classification". A group whose evaluation value is less than the lower threshold is referred to as "lower classification".

  After the above classification, it is referred to as any one index (hereinafter, "focused index"). One trial is extracted as a characteristic trial in the second embodiment if there is only one trial belonging to the upper classification or the lower classification based on the above. On the other hand, when a plurality of trials belong to the upper classification or the lower classification, it is determined whether or not the plurality of trials belong to the same group as each other with respect to other indicators. That is, with regard to the other indexes, it is determined whether the relationship between the upper threshold and the lower threshold is common. Trials belonging to a category different from any one of the plurality of trials with respect to the other indicators are extracted as characteristic trials in the second embodiment. On the other hand, when there are a plurality of trials belonging to the same category mutually for the other indicators, among the plurality of trials, the trials having the largest absolute value of the difference between the upper threshold or the lower threshold for the evaluation value regarding the target indicator , Extracted as a characteristic trial in the second embodiment. However, in this case, the number of trials to be extracted may not be limited to one. For example, the absolute value of the difference with the upper threshold or the lower threshold may be extracted up to the Nth trial. However, N is a value smaller than the number of trials which belong to the same classification mutually also about other indices. Note that the other indicators may not be all the other indicators. For example, when the number of other indicators is 10, it may be determined whether or not the other nine indicators belong to the same group.

  For example, in FIG. 11, evaluation values of trials belonging to the upper classification are surrounded by thick lines. The upper threshold for index 1 is 200 and the lower threshold is 0. The upper threshold for index 2 is 30, and the lower threshold is 0.

  Assuming that the indicator 1 is a focused indicator, with regard to the measure 1, the trial concerning the environmental characteristic 2 and the trial concerning the environmental characteristic 3 belong to the upper classification. However, these two trials are different from each other in the group to which the index 2 belongs. Thus, in this case, both trials are extracted as characteristic trials. Moreover, regarding the measure 2 of the index 1, the trial concerning the environmental characteristic 1 and the trial concerning the environmental characteristic 4 belong to the above classification. These two trials also belong to the same group (middle classification) with respect to index 2. That is, it is not determined whether or not the trial 1 belongs to the same classification as the index 1, but it is determined in the indicator 2 whether the trials belong to the same classification. In this case, among these two trials, the trial of the environmental characteristic 4 having a large absolute value of the difference with the threshold value above the index 1 is extracted as the characteristic trial.

  Note that characteristic trials are extracted also for trials belonging to the lower classification. Further, the index 2 is similarly set as the target index, and extraction of characteristic trials is performed.

  FIG. 12 is a diagram showing an example of a functional configuration of the extraction device in the second embodiment. In FIG. 12, the same parts as those in FIG. 5 are indicated by the same reference numerals and the explanation thereof will be omitted. Further, in FIG. 12, the parts corresponding to those in FIG. 5 are denoted by the reference numerals with “a” added at the end of the reference numerals in FIG.

  In FIG. 12, the extraction device 10a includes a result acquisition unit 11a, a classification unit 17, an extraction unit 18, an output unit 15, and the like. These units are realized by processing that one or more programs installed in the extraction device 10 a cause the CPU 104 to execute. The extraction device 10a also uses a simulation result storage unit 16a. The simulation result storage unit 16a can be realized by using the auxiliary storage device 102 or a storage device connected to the extraction device 10a via a network.

  The simulation result storage unit 16 a stores, for each trial of the simulation, evaluation values output by the trial for a plurality of types of indices. The result acquisition unit 11a acquires the simulation result from the simulation result storage unit 16a.

  The classification unit 17 is described in FIG. 11 for each of a plurality of trials, based on the comparison of the evaluation value acquired by the fruit acquiring unit 11a with respect to the relevant index and the threshold value for the relevant indicator, for each index type. Classification. In other words, the classification unit identifies the trials belonging to the upper classification and the trials belonging to the lower classification according to the type of index.

  The extraction unit 18 extracts trials classified into groups different from any other trial with respect to a certain index, among trial groups relating to the same measure. That is, when one trial belongs to the upper classification or the lower classification, the trial is extracted. In addition, when there is a plurality of trials classified into the upper classification or the lower classification with respect to a certain index among the trial groups according to the same measure, the extraction unit 18 performs one trial of the plurality of trials. Extract. Among the trial groups classified into the same group with respect to all other indicators other than the certain indicator, the one trial is the difference between the evaluation value for the certain indicator and the upper threshold or the lower threshold. Is the trial whose absolute value is the largest.

  FIG. 13 is a flowchart for explaining an example of a processing procedure of classification processing of trials in the second embodiment.

  In step S201, the result acquisition unit 11a acquires a simulation result from the simulation result storage unit 16a.

  FIG. 14 is a diagram showing an example of a simulation result of the second embodiment. FIG. 14 shows the evaluation values output by 15 trials performed on five environmental characteristics for each of the three types of measures. (1) shows an evaluation value concerning index 1, and (2) shows an evaluation value concerning index 2. That is, FIG. 14 shows an example in which evaluation values regarding two types of indexes of index 1 and index 2 are output in one trial.

  Subsequently, the classification unit 17 substitutes 1 into the variable i (S202). The variable i is a variable for identifying an index to be processed. Subsequently, the classification unit 17 determines whether the value of the variable i is equal to or less than the number of types of index (here, 2) (S203).

  If the value of the variable i is equal to or less than the number of types of indicators (Yes in S203), the classification unit 17 substitutes 1 into the variable j (S204). The variable j is a variable for identifying the measure to be processed. Subsequently, the classification unit 17 determines whether the value of the variable j is equal to or less than the number of measures (S205). In the example of FIG. 14, the number of measures is three.

  If the value of the variable j is equal to or less than the number of measures (Yes in S205), the classification unit 17 substitutes 1 for the variable k (S206). The variable k is a variable for identifying an environmental characteristic to be processed. Subsequently, the classification unit 17 determines whether the value of the variable k is equal to or less than the number of environmental characteristics (S207). In the example of FIG. 14, the number of environmental characteristics is five.

  If the value of the variable k is less than or equal to the number of environmental characteristics (Yes in S207), the classification unit 17 substitutes the upper threshold for the index i for the measure j into the variable T (i, j) to set the variable t (i, j). Substitute the lower threshold for index i relating to measure j into j) (S208). The upper threshold and the lower threshold may be different not only for each index but also for each measure. The upper threshold and the lower threshold are preset. Alternatively, in step S208, the average value and the standard deviation of the indices i of all the trials for the measure j may be calculated, and the upper threshold and the lower threshold may be determined by the following operation.

Upper threshold = average value + standard deviation. Lower threshold = average value−standard deviation. Then, the classification unit 17 calculates the evaluation value of the index j of the trial relating to the measure j and the environmental characteristic k, V (i, j, k ) And T (i, j) and t (i, j). The classification unit 17 substitutes identification information (hereinafter referred to as "group number") of the group to which the trial is classified, for the variable C (i, j, k) according to the comparison result (S209). . That is, the variable C (i, j, k) is a variable for storing the group number of the group to be classified based on the evaluation value of the index i for the trial related to the measure j and the environmental characteristic k. “1” as a group number indicates the above classification. "2" as a group number indicates a lower classification. "3" as a group number indicates middle classification. Therefore, if the evaluation value is larger than the upper threshold (ie, if V (i, j, k)> T (i, j)), 1 is substituted into variable C (i, j, k) Ru. Or, if the evaluation value is smaller than the lower threshold (that is, if V (i, j, k) <t (i, j)), 2 is substituted for the variable C (i, j, k) . Alternatively, if the evaluation value is equal to or lower than the upper threshold and equal to or higher than the lower threshold, 3 is substituted for the variable C (i, j, k).

  Subsequently, the classification unit 17 adds 1 to the variable k (S210), and repeats step S207 and subsequent steps. In the process in which step S207 and subsequent steps are repeated, if the value of variable k becomes larger than the number of environmental characteristics, that is, if step S208 and subsequent steps are executed for all environmental characteristics (No in S207), classification unit 17 selects variable j 1 is added to (S211), and step S205 and subsequent steps are repeated.

  In the process of repeating step S205 and subsequent steps, if the value of variable j becomes larger than the number of measures, that is, if step S206 and subsequent steps are executed for all the measures (No in S205), classification unit 17 sets variable i to 1 Are added (S212), and step S203 and subsequent steps are repeated. When the value of the variable i becomes larger than the number of index types in the process of repeating step S203 and subsequent steps, that is, when step S204 and subsequent steps are executed for all the indices (No in S203), the processing of FIG. .

  Therefore, in the process of FIG. 13, classification is performed for each index for all trials, and the classification result is substituted into the variable C (i, j, k).

  FIG. 15 is a diagram showing an example of classification results of each trial based on the evaluation value of each index. FIG. 15 shows the classification results of each trial based on the evaluation values shown in FIG. That is, FIG. 15 shows the group numbers of the groups to be classified for each trial. In the example of FIG. 15, the upper threshold is 300 and the lower threshold is 100 for both the index 1 and the index 2.

  If the trial concerning the j-th measure and the k-th environmental characteristic is expressed as trial (j, k), trial (1, 2) and trial (1, 5) are group number 1 (index) for index 1. That is, it is classified into the above classification). Also, with regard to index 2, trials (2, 1), trials (1, 2), and trials (2, 3) are classified into group number 2 (that is, lower classification).

  Subsequently, a processing procedure performed by the extraction unit 18 after the processing of FIG. 13 will be described. FIG. 16 is a flowchart for describing an example of a processing procedure of trial extraction processing in the second embodiment.

  In step S301, the extraction unit 18 initializes the variable E to be empty. The variable E is a variable for storing identification information of the finally extracted trial. Subsequently, the extraction unit 18 substitutes 1 into the variable i and initializes the variable E (i) to empty (S302). The meanings of the variables i, j and k are the same as in FIG. The variable E (i) is a variable for storing identification information of trials that have been made extraction candidates in the trial to be processed.

  Subsequently, the extraction unit 18 determines whether the value of the variable i is equal to or less than the number of types of indices (S303). If the value of the variable i is equal to or less than the number of types of indicators (Yes in S303), the extraction unit 18 substitutes 1 into the variable j (S304). Subsequently, the extraction unit 18 determines whether the value of the variable j is equal to or less than the number of measures (S305). If the value of the variable j is equal to or less than the number of measures (Yes in S305), the extraction unit 18 substitutes 1 into the variable k (S306). Subsequently, the extraction unit 18 determines whether the value of the variable k is less than or equal to the number of environmental characteristics (S307).

  When the value of the variable k is equal to or less than the number of environmental characteristics (Yes in S307), the extraction unit 18 executes an extraction candidate update process (S308). In the extraction candidate updating process, the content of the variable E (i) is updated. For example, if trial (j, k) is determined as an extraction candidate, identification information of trial (j, k) is substituted into variable (i). In the present embodiment, the identification information is expressed in the form of <j, k>.

  Subsequently, the extraction unit 18 adds 1 to the variable k (S309), and repeats step S307 and subsequent steps. If the value of the variable k becomes larger than the number of environmental characteristics in the process of repeating step S307 and subsequent steps, that is, if step S308 is executed for all the environmental characteristics (No in S307), the extraction unit 18 sets the variable j as the variable j. 1 is added (S310), and step S305 and subsequent steps are repeated. In the process of repeating step S305 and subsequent steps, when the value of variable j becomes larger than the number of measures, that is, step S306 and subsequent steps are executed for all the measures (No in S305), extraction unit 18 sets variable E to A union of the contents of the variable E and the contents of the variable E (i) is substituted, 1 is added to the variable i (S311), and the steps after S303 are repeated. When the value of the variable i becomes larger than the number of index types in the process of repeating step S303 and subsequent steps, that is, when step S304 and subsequent steps are executed for all the indices (No in S303), the output unit 15 performs variable E The contents of are output (S312).

  Subsequently, the details of step S308 will be described. FIG. 17 is a flowchart for explaining an example of the processing procedure of the extraction candidate updating process.

  In step S401, the extraction unit 18 determines whether C (i ′, j, k) <3 holds. Here, i ′ is one or more values smaller than the value of the variable i at the time when step S308 of FIG. 16 is executed. Also, j and k are the values of variables j and k at that time. That is, in step S401, it is determined whether the trial (j, k) belongs to the upper classification or the lower classification with respect to the index i ′ (any index before the index i). The determination means determination as to whether or not trial (j, k) has been determined as the target of extraction in the process of FIG. 17 executed for the index i ′. That is, step S401 is a determination process for preventing extraction twice or more for the same trial (j, k).

  If C (i ′, j, k) <3 is satisfied (No in S401), the process of FIG. 17 ends. When C (i ′, j, k) <3 does not hold (Yes in S401), the extraction unit 18 branches the process based on the value of C (i, j, k) (S402).

  When the value of C (i, j, k) is 1 (“1” in S 402), the extraction unit 18 determines whether C (n, j, k) = C (n, j, k ′) holds It is determined whether or not it is (S403). Here, n is an arbitrary index (ie, all indexes). K ′ is an environmental characteristic already stored in E (i) for the measure j. That is, any trial (j, k ') and trial (j, k) which have the same policy as the trial (j, k) and have already been selected as extraction candidates for the index i are the same group for all the indicators It is determined whether or not it is classified into.

  When C (n, j, k) = C (n, j, k ′) does not hold (No in S403), the extraction unit 18 identifies the identification information of trial (j, k) for E (i). <J, k> is added (S404). That is, if trial (j, k) is classified into different groups with respect to trial (j, k ') and all the indicators, or if E (i) is empty, etc., trial (j, k) Is a candidate for extraction.

  If C (n, j, k) = C (n, j, k ') holds (Yes in S403), the extraction unit 18 determines that V (i, j, k)> V (i, j, k'). ) Is determined (S405). That is, whether or not <j, k ′> is already stored in E (i), and if it is stored, the trial (j, k ′) is more trial than the evaluation value according to the index i of the trial (j, k ′). , K) is judged whether the evaluation value concerning the index i is larger. In addition, the said judgment is strictly the evaluation value which concerns on the index i of trial (j, k ')-the evaluation value which concerns on the index i of trial (j, k) rather than the absolute value of the threshold value on This corresponds to the determination of whether the absolute value of the threshold is larger.

  If k ′ satisfying V (i, j, k)> V (i, j, k ′) exists (Yes in S 405), the extraction unit 18 removes <j, k ′> from E (i). Then, <j, k> is added to E (i) (S406). If k ′ satisfying V (i, j, k)> V (i, j, k ′) does not exist (No in S405), <j, k> is not added to E (i).

  On the other hand, when the value of C (i, j, k) is 2 (“2” in S402), the extraction unit 18 performs the same determination as in step S403 in step S407. That is, any trial (j, k ') and trial (j, k) which have the same policy as the trial (j, k) and have already been selected as extraction candidates for the index i are the same group for all the indicators It is determined whether or not it is classified into.

  When C (n, j, k) = C (n, j, k ′) does not hold (No in S407), the extraction unit 18 identifies the identification information of trial (j, k) for E (i). <J, k> is added (S408). If C (n, j, k) = C (n, j, k ') holds (Yes in S407), the extraction unit 18 determines that V (i, j, k) <V (i, j, k'). ) Is determined (S409). That is, whether or not <j, k ′> is already stored in E (i), and if it is stored, the trial (j, k ′) is more trial than the evaluation value according to the index i of the trial (j, k ′). , K) is determined whether the evaluation value according to index i is smaller. In addition, the said determination is strictly the evaluation value which concerns on the index i of trial (j, k ')-the evaluation value which concerns on the index i of trial (j, k) rather than the absolute value of the lower threshold. This corresponds to the determination of whether the absolute value of the threshold is smaller.

  If k ′ satisfying V (i, j, k) <V (i, j, k ′) exists (Yes in S409), the extraction unit 18 removes <j, k ′> from E (i). Then, <j, k> is added to E (i) (S410). If k ′ satisfying V (i, j, k) <V (i, j, k ′) does not exist (No in S409), <j, k> is not added to E (i).

  Further, when the value of C (i, j, k) is 3 ("3" in S402), the update of E (i) is not performed. In the present embodiment, trials belonging to middle classification are not considered extraction candidates.

  As described above, according to the second embodiment, a trial related to a value (value belonging to the upper classification or the lower classification) in which the evaluation value of any of the plurality of indices deviates from the average is extracted. Therefore, compared with the case where trial is extracted based on one index, it is possible to reduce the possibility of missing the risk behind the measure.

  In addition, when there are a plurality of trials classified into the same group mutually for all the indicators, one trial of the plurality of trials is set as an extraction target. As a result, a plurality of trials likely to exhibit the same phenomenon can be narrowed down to one, and the number of trials extracted can be reduced to an analysis possible level. A specific example is given and demonstrated about this point. For example, in the simulation example of cash register waiting time in a store, in the case of short-term congestion, the maximum cash waiting time of each trial increases, and the average cash waiting time of each trial is average. It is considered that there is a high possibility of becoming a value. On the other hand, in the case of long-term or chronic congestion, it is possible that both values become large for each trial. For example, if trials (j, k) and trials (j, k + 1) show the same tendency in both maximum and average waiting time of cash register (if they are classified into the same group) Even if both (j, k) and trial (j, k + 1) are analyzed, the cause of the increase in the maximum value and the cause of the increase in the average value are likely to be the same. Therefore, in this case, it is considered sufficient that either one of the trial (j, k) and the trial (j, k + 1) is extracted as an analysis target. In addition, the one whose evaluation value deviates more from the average (that is, the one in which the phenomenon appears significantly) is suitable as the analysis target. On the other hand, if trial (j, k) and trial (j, k + 1) show the same tendency with regard to the maximum value of cash register latency and show different tendencies with respect to the average value The phenomena shown by trial (j, k) and trial (j, k + 1) are likely to be different. Therefore, in this case, both trials are extracted in the present embodiment.

  Next, the third embodiment will be described. In the third embodiment, points different from the second embodiment will be described. The third embodiment may be similar to the second embodiment in the points not particularly mentioned.

  FIG. 18 is a diagram showing an example of a functional configuration of the extraction device in the third embodiment. In FIG. 18, the same parts as in FIG. 12 or FIG.

  The extraction device 10b according to the third embodiment further includes a normalization unit 12 and a difference calculation unit 13. The processes executed by the normalization unit 12 and the difference calculation unit 13 are as described in the first embodiment. However, in the third embodiment, the normalization unit 12 normalizes the evaluation value for each index. The difference calculation unit 13 calculates, for each index, an average value for each environmental common evaluation value group for a set of normalized evaluation values, and for each environmental common evaluation value group, a difference from the average value of each evaluation value calculate.

  The classification unit 17 classifies the output result of the difference calculation unit 13 based on the upper threshold and the lower threshold. In this case, since the normalization is performed with respect to the evaluation values of all the indices, the upper threshold and the lower threshold may be common to all the indices. For example, the upper threshold may be +1 and the lower threshold may be -1.

  Furthermore, as in the first embodiment, normalization by the normalization unit 12 may be omitted.

  The description of the processing procedure executed by the extraction device 10b in the third embodiment will be omitted as it is obvious from the first embodiment and the second embodiment.

  In each of the above embodiments, the extraction device 10 is an example of an information processing device. The evaluation value is an example of the output value. The result acquisition unit 11 is an example of the acquisition unit of claim 6. The difference calculation unit 13 is an example of a calculation unit of claim 6. The result acquisition unit 11a is an example of the acquisition unit of claim 7. The classification unit 17 is an example of the specification unit of claim 7. The extraction unit 18 is an example of the extraction unit of claim 7.

  Although the embodiments of the present invention have been described above in detail, the present invention is not limited to such specific embodiments, and various modifications may be made within the scope of the present invention as set forth in the claims.・ Change is possible.

10, 10a, 10b Extraction Device 11, 11a Result Acquisition Unit 12 Normalization Unit 13 Difference Calculation Unit 14 Extraction Unit 15 Output Unit 16, 16a Simulation Result Storage Unit 17 Classification Unit 18 Extraction Unit 100 Drive Device 101 Recording Medium 102 Auxiliary Storage Device 103 Memory Unit 104 CPU
105 Interface device B bus

Claims (12)

For each of a plurality of measures for the subject of the simulation, a plurality of each of the measures and each of the characteristics obtained by executing the simulation trial for a plurality of characteristics of elements having uncertainty with respect to the measures Processing to obtain the output value of each trial,
Among the plurality of trials, for each first trial group common to the characteristics, the difference between the average value of the output value group according to the first trial group and each output value included in the output value group The process to calculate
A process of extracting a partial trial from the plurality of trials based on the difference calculated for each output value of the plurality of trials;
An extraction method characterized in that the computer executes. The extracting process extracts a trial related to an output value in which the difference exceeds a threshold.
The extraction method according to claim 1, characterized in that: Among the plurality of trials, for each output value group pertaining to each of the second trial group common to the measures, the difference between the plurality of measures of the indicator indicating the variation of the output value group is reduced The computer executes a process of changing each output value included in the output value group,
The calculating process calculates the difference for the value changed by the changing process.
The extraction method according to claim 1 or 2, characterized in that:   The extraction method according to any one of claims 1 to 3, wherein the computer executes a process of outputting a measure and a characteristic corresponding to the extracted trial. A process of acquiring values output for each of a plurality of trials for operating a virtual subject in each of a plurality of environments built on a computer;
A process of calculating, for each value included in the value group, a difference from the average value of the value group pertaining to the trial group, for each trial group having the same environment among the plurality of trials ;
Extracting a portion of the plurality of trials based on the difference calculated for each value of the trials;
An extraction method characterized in that the computer executes. The computer is
For each of a plurality of measures for the subject of the simulation, a plurality of each of the measures and each of the characteristics obtained by executing the simulation trial for a plurality of characteristics of elements having uncertainty with respect to the measures Get output values for each of the multiple types of indicators in the trial,
For each of the indicators, specify an attempt whose output value for the indicator is greater than a first threshold for the indicator or less than a second threshold for the indicator,
When a plurality of trials are specified for any one of the measures with respect to a first indicator among the plurality of types of indicators, the first threshold and the second threshold with respect to other indicators among the plurality of trials. Extract some trials out of the plurality of trials out of trials having a common relationship with the threshold value,
An extraction method characterized by performing processing. The extracting process extracts a trial in which an absolute value of a difference between an output value related to the first index and the first threshold or the second threshold with respect to the first index is largest.
The extraction method according to claim 6 , characterized in that. Gets the value output for a plurality types of indicators virtual entity for each attempt to operate on the basis of a plurality of measures for each of the plurality of environment built on a computer,
For each of the indicator, the relationship between the value related to the metrics and a plurality of threshold values to identify a predetermined condition is satisfied trials,
When a plurality of trials is specified for any one of the measures with respect to the first indicator among the plurality of indicators, another trial among the plurality of trials is a trial in which the relationship with the plurality of threshold values is common with respect to the other indicator. Extract some of the multiple trials out of
An extraction method characterized in that a computer executes a process. For each of a plurality of measures for the subject of the simulation, a plurality of each of the measures and each of the characteristics obtained by executing the simulation trial for a plurality of characteristics of elements having uncertainty with respect to the measures An acquisition unit that acquires the output value of each trial;
Among the plurality of trials, for each first trial group common to the characteristics, the difference between the average value of the output value group according to the first trial group and each output value included in the output value group A calculation unit to calculate
An extraction unit that extracts a part of trials from the plurality of trials based on the difference calculated for each output value of each of the plurality of trials;
An information processing apparatus comprising: For each of a plurality of measures for the subject of the simulation, a plurality of each of the measures and each of the characteristics obtained by executing the simulation trial for a plurality of characteristics of elements having uncertainty with respect to the measures An acquisition unit for acquiring output values of each of a plurality of types of indicators of trial;
A specifying unit that specifies, for each of the indicators, an attempt in which an output value regarding the indicator is larger than a first threshold for the indicator or smaller than a second threshold for the indicator;
When a plurality of trials are specified for any one of the measures with respect to a first indicator among the plurality of types of indicators, the first threshold and the second threshold with respect to other indicators among the plurality of trials. An extraction unit that extracts some of the plurality of trials out of the trials having a common relationship with the threshold;
An information processing apparatus comprising: For each of a plurality of measures for the subject of the simulation, a plurality of each of the measures and each of the characteristics obtained by executing the simulation trial for a plurality of characteristics of elements having uncertainty with respect to the measures Processing to obtain the output value of each trial,
Among the plurality of trials, for each first trial group common to the characteristics, the difference between the average value of the output value group according to the first trial group and each output value included in the output value group The process to calculate
A process of extracting a partial trial from the plurality of trials based on the difference calculated for each output value of the plurality of trials;
An extraction program that causes a computer to execute. On the computer
For each of a plurality of measures for the subject of the simulation, a plurality of each of the measures and each of the characteristics obtained by executing the simulation trial for a plurality of characteristics of elements having uncertainty with respect to the measures Get output values for each of the multiple types of indicators in the trial,
For each of the indicators, specify an attempt whose output value for the indicator is greater than a first threshold for the indicator or less than a second threshold for the indicator,
When a plurality of trials are specified for any one of the measures with respect to a first indicator among the plurality of types of indicators, the first threshold and the second threshold with respect to other indicators among the plurality of trials. Extract some trials out of the plurality of trials out of trials having a common relationship with the threshold value,
An extraction program that runs a process.

Images