JP7756579B2 - Computer system, method for evaluating policy definition, and program - Google Patents

Description

The present invention relates to a technique for assessing bias in factors included in a comparison group used to evaluate a policy.

Various measures are implemented to improve indicators such as business efficiency, productivity, and sales. In this specification, a "measure" is defined as an action aimed at achieving a certain effect, using resources at the target of the measure.

AB testing is known as a method for evaluating policies. In AB testing, people are divided into two comparison groups, A and B, and different actions are performed on groups A and B. The results of the actions in groups A and B are compared to evaluate the effectiveness and causal relationships of the policy.

In A/B testing, the allocation of elements to comparison groups is important. The measures described in this specification require appropriate allocation of targets and resources. In response to this, a method for allocating comparison groups based on the attributes of the target to be divided is known (see, for example, paragraphs [0046] and [0047] of Patent Document 1).

Japanese Patent Application Laid-Open No. 2021-89485

Conventional techniques focus on bias in the attributes of elements between comparison groups, but do not address bias in attributes between the set of elements to be selected (population) and the group of elements included in the comparison group. It is desirable that the distribution of attributes of elements included in the comparison group be similar to the distribution of attributes of elements included in the population. If the two distributions are different, this indicates the occurrence of bias.

The present invention provides a system and method for assessing the bias of elements included in a comparison group relative to a population of elements.

A representative example of the invention disclosed in the present application is as follows: That is, a computer system including at least one computer having a processor and a storage device connected to the processor, holds policy definition management information that stores policy definition data including allocation conditions for generating a comparison group, which is a set of elements of a policy, to be used in evaluating the effectiveness of the policy, and the at least one computer identifies the comparison group generated based on the policy definition data and a population of the elements that meets the allocation conditions included in the policy definition data, calculates a similarity between a distribution of attributes of the elements included in the comparison group and a distribution of attributes of the elements included in the population, and determines whether or not bias of the elements in the comparison group has occurred based on the similarity .

According to the present invention, a computer system can evaluate and present the bias of elements included in a comparison group relative to the population of elements. Other issues, configurations, and advantages will become clearer from the description of the following examples.

FIG. 1 is a diagram illustrating an example of the configuration of a computer system according to a first embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer included in the policy evaluation system of the first embodiment. FIG. 10 is a diagram illustrating an example of policy definition management information according to the first embodiment. FIG. 10 is a diagram illustrating an example of resource management information according to the first embodiment. FIG. 10 is a diagram illustrating an example of policy target management information according to the first embodiment. FIG. 10 is a diagram illustrating an example of policy implementation plan management information according to the first embodiment. FIG. 10 is a diagram illustrating an example of bias analysis result management information according to the first embodiment. FIG. 10 is a diagram illustrating an example of bias analysis result management information according to the first embodiment. FIG. 10 is a diagram illustrating an example of a screen displayed by the terminal according to the first embodiment. 10 is a flowchart illustrating an example of a policy implementation plan generation process executed by the policy evaluation system according to the first embodiment. 10 is a flowchart illustrating an example of a bias evaluation index calculation process executed by the policy evaluation system according to the first embodiment. 10 is a flowchart illustrating an example of a bias evaluation index calculation process executed by the policy evaluation system according to the first embodiment. FIG. 11 is a diagram illustrating an example of second bias analysis information according to the second embodiment. FIG. 10 is a diagram illustrating an example of a screen displayed by the terminal according to the first embodiment. 10 is a flowchart illustrating an example of a factor analysis process executed by the policy evaluation system according to the second embodiment. FIG. 10 is a diagram illustrating an example of a screen displayed by a terminal according to a second embodiment. FIG. 11 is a diagram illustrating an example of a screen displayed by a terminal according to a third embodiment. 11 is a flowchart illustrating an example of a policy definition evaluation process executed by the policy evaluation system according to the third embodiment. FIG. 1 is a diagram illustrating a conventional problem.

The following describes an embodiment of the present invention with reference to the accompanying drawings. However, the present invention should not be construed as being limited to the description of the embodiment shown below. Those skilled in the art will readily understand that the specific configuration can be modified without departing from the spirit or intent of the present invention.

In the configuration of the invention described below, identical or similar components or functions are designated by the same reference numerals, and duplicate explanations will be omitted.

The terms "first," "second," "third," etc. used in this specification are used to identify components and do not necessarily limit the number or order.

In this specification, the comparison groups for evaluating policies are defined as Group A and Group B. Furthermore, when there is no distinction between targets and resources, they are referred to as elements.

Figure 1 is a diagram illustrating an example of the configuration of a computer system according to Example 1. Figure 2 is a diagram illustrating an example of the hardware configuration of a computer included in the policy evaluation system according to Example 1.

The computer system consists of a policy evaluation system 100 and a terminal 101. The policy evaluation system 100 and the terminal 101 are connected to each other via a network 102 such as a WAN (Wide Area Network) or a LAN (Local Area Network). The network 102 may be connected via either a wired or wireless method.

The policy evaluation system 100 generates a policy implementation plan and evaluates the bias of the comparison group used in the policy evaluation, and is composed of a computer 200 as shown in Figure 2.

The computer 200 has a processor 201, a main memory device 202, a secondary memory device 203, and a network interface 204. Each hardware element is connected to each other via a bus. The computer 200 may also have input devices such as a keyboard, mouse, and touch panel, and output devices such as a display and printer.

The processor 201 executes programs stored in the main memory device 202. By executing processing in accordance with the programs, the processor 201 operates as a functional unit (module) that realizes a specific function. In the following explanation, when processing is described using a functional unit as the subject, this indicates that the processor 201 is executing a program that realizes that functional unit.

The main memory device 202 is a storage device such as a DRAM (Dynamic Random Access Memory) that stores programs executed by the processor 201 and data used by the programs. The main memory device 202 is also used as a work area. The secondary memory device 203 is a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that permanently stores data.

The programs and data stored in the main memory device 202 may also be stored in the secondary memory device 203. In this case, the processor 201 reads the programs and data from the secondary memory device 203 and loads them into the main memory device 202.

The policy evaluation system 100 has a policy planning unit 110, a bias analysis unit 111, and a policy definition evaluation unit 112. The policy evaluation system 100 also holds policy definition management information 120, resource management information 121, policy target management information 122, policy implementation plan management information 123, and bias analysis result management information 124.

Measure definition management information 120 is information for managing data including measure definitions (measure definition data). Here, a measure definition is a concept that includes the specific content of a measure and the allocation conditions for a comparison group for evaluating the measure. Resource management information 121 is information for managing the resources used in a measure. Measure target management information 122 is information for managing the targets of a measure. Measure implementation plan management information 123 is information for managing the implementation plan for a measure. Bias analysis result management information 124 is information for managing the results of bias analysis of a comparison group.

The policy planning unit 110 generates an implementation plan based on the policy definition data and registers the generated implementation plan in the policy implementation plan management information 123. When generating implementation plans for multiple policies, the policy planning unit 110 in this embodiment allocates elements to comparison groups in a way that prevents bias in elements in groups A and B, and bias in elements included in the comparison group relative to the set of elements (population) that meet the allocation conditions for the comparison group. Details will be provided below.

In the following description, the bias of elements in groups A and B will be referred to as the first bias, and the bias of elements included in the comparison group relative to the set of elements that meet the allocation conditions of the comparison group will be referred to as the second bias.

The bias analysis unit 111 analyzes various biases in the comparison group. The bias analysis unit 111 includes an evaluation index calculation unit 130 and a factor analysis unit 131. The evaluation index calculation unit 130 calculates an index representing the intensity of occurrence of each of the first bias and the second bias. The factor analysis unit 131 analyzes the factors causing the second bias. Details of the factor analysis unit 131 will be explained in Example 2.

The policy definition evaluation unit 112 evaluates the likelihood of the new policy causing the second bias based on the policy definition data of the new policy. Details of the policy definition evaluation unit 112 will be explained in Example 3.

Regarding the functional units of the policy evaluation system 100, multiple functional units may be combined into a single functional unit, or a single functional unit may be divided into multiple functional units for each function.

The terminal 101 is operated by a user who uses the policy evaluation system 100. The terminal 101 has a processor, main memory device, network interface, input device, and output device (not shown). The terminal 101 has a data reception unit 140 and a display unit 141. The data reception unit 140 receives data input from the user. The display unit 141 displays various information.

The second bias occurs significantly when multiple measures are implemented simultaneously. Figure 17 is a diagram illustrating the problems with the conventional approach. Here, we will explain the example of grouping employees (resources) in call center operations. In Figure 17, circles represent employees. Consider the following resource grouping for measures 1 and 2.
Measure 1 Group A: Measure 1 for women Group B: Measure 2 for men Group A: Measure 2 for people in their 30s Group B: Measure for people in their 20s

As shown in Figure 17, when grouping is performed for each measure in the order of Measure 1 and Measure 2, Group B for Measure 2 contains only female employees. However, the set of elements (population) that meets the allocation conditions for Group B for Measure 2 includes men. Therefore, it can be determined that the second bias has occurred in Group B for Measure 2.

Figure 3 is a diagram showing an example of the policy definition management information 120 in Example 1. The policy definition management information 120 stores entries including a policy ID 301, an action 302, a customer allocation condition 303, and a resource allocation condition 304. There is one entry for each policy.

Measure ID 301 is a field that stores identification information for uniquely identifying a measure. Action 302 is a field that stores the action of the measure. Customer allocation conditions 303 are a group of fields that store the allocation conditions of comparison groups related to customers who are the target of the measure. The customer allocation conditions 303 store the allocation conditions of groups A and B. Resource allocation conditions 304 are a group of fields that store the allocation conditions of comparison groups related to resources. The resource allocation conditions 304 store the allocation conditions of groups A and B.

Figure 4 is a diagram showing an example of resource management information 121 in Example 1. The resource management information 121 stores entries including a resource ID 401 and attributes 402. There is one entry for each resource.

Resource ID 401 is a field that stores identification information for uniquely identifying a resource. Attributes 402 are a group of fields that store resource attributes. If the resource is an employee, attributes 402 store fields such as staff ID, age, gender, and working hours. Note that the present invention is not limited to the fields included in attributes 402.

Figure 5 is a diagram showing an example of policy target management information 122 in Example 1. The policy target management information 122 stores entries including a target ID 501 and attributes 502. There is one entry for each target.

Object ID 501 is a field that stores identification information for uniquely identifying an object. Attributes 502 are a group of fields that store the attributes of the object. If the object is a customer, attributes 502 store fields such as age, gender, and annual income. Note that the present invention is not limited to the fields included in attributes 502.

Figure 6 is a diagram showing an example of the policy implementation plan management information 123 in Example 1. The policy implementation plan management information 123 stores entries including a policy ID 601, an AB type 602, a target list 603, and a resource list 604. There is one entry for each combination of policy and comparison group type.

Measure ID 601 is the same field as Measure ID 301. AB Type 602 is a field that stores the type of comparison group. AB Type 602 stores either "A" representing Group A or "B" representing Group B. Target List 603 is a field that stores a list of targets assigned to the comparison group of the measure. Resource List 604 is a field that stores a list of resources assigned to the comparison group of the measure.

Figures 7A and 7B are diagrams showing an example of bias analysis result management information 124 in Example 1. The bias analysis result management information 124 includes first bias analysis information 700 and second bias analysis information 710.

The first bias analysis information 700 stores entries including a policy ID 701 and an A/B group bias 702. There is one entry for each policy.

Measure ID 701 is the same field as Measure ID 301. Group AB bias 702 is a field that stores an index (first bias index) that represents the intensity of the first bias.

The second bias analysis information 710 stores entries including a policy ID 711, an AB type 712, and a distribution bias 713. There is one entry for each combination of a policy and a comparison group type.

Measure ID 711 is the same field as Measure ID 301. AB Type 712 is the same field as AB Type 602. Distribution Bias 713 is a group of fields that store an index (second bias index) that represents the intensity of occurrence of the second bias. Distribution Bias 713 includes a group of fields that store the intensity of occurrence of the second bias for each attribute of the target, and a group of fields that store the intensity of occurrence of the second bias for each attribute of the resource.

Figure 8 is a diagram showing an example of a screen displayed by the terminal 101 in Example 1. The display unit 141 of the terminal 101 displays a screen 800 for registering policy definition data.

The screen 800 includes a group A setting area 810, a group B setting area 820, a manual condition setting area 830, a policy ID setting field 840, and a registration button 850.

The group A setting area 810 includes fields for setting the allocation conditions for resources and targets in group A and the actions in group A. The group B setting area 820 includes fields for setting the allocation conditions for resources and targets in group B and the actions in group B. The manual condition setting area 830 includes fields for setting other conditions. Fields are added by pressing the add button. The policy ID setting field 840 is a field for setting identification information for the policy. The register button 850 is an operation button for registering policy definition data.

The user makes various settings on screen 800 and presses the register button 850. The data reception unit 140 of the terminal 101 sends the policy definition data, including the various settings on screen 800, to the policy evaluation system 100. The policy evaluation system 100 registers the policy definition data in the policy definition management information 120.

Figure 9 is a flowchart illustrating an example of the policy implementation plan generation process executed by the policy evaluation system 100 of Example 1.

When the policy evaluation system 100 receives an instruction to generate a policy implementation plan from the terminal 101, it starts the processing described below.

The policy planning unit 110 identifies the policy to be implemented by referring to the policy definition management information 120 (step S101). Specifically, the policy planning unit 110 obtains each entry in the policy definition management information 120 and generates a list of policies. The policy planning unit 110 also adds entries for group A and group B for each policy to the policy implementation plan management information 123.

For each policy, the policy planning unit 110 extracts a set of targets (target population) that meet the target allocation conditions for groups A and B, and extracts a set of resources (resource population) that meet the resource allocation conditions for groups A and B (step S102). Here, for each policy, a target population for group A, a target population for group B, a resource population for group A, and a resource population for group B are extracted.

The policy planning unit 110 starts a policy loop process (step S103). Specifically, the policy planning unit 110 selects one policy from the list of policies.

The policy planning unit 110 selects targets to be allocated to Group A and targets to be allocated to Group B from the target population of the selected policy, and selects resources to be allocated to Group A and resources to be allocated to Group B from the resource population of the selected policy (step S104). The targets and resources are selected randomly from each population. The policy planning unit 110 sets identification information for the targets and resources in the target list 603 and resource list 604 of each entry in the policy implementation plan management information 123.

The policy planning unit 110 determines whether processing has been completed for all policies (step S105). If processing has not been completed for all policies, the policy planning unit 110 selects a new policy and returns to step S104.

In this way, by selecting one element from the population for each measure, it is possible to suppress the occurrence of the second bias caused by selecting elements all at once. Furthermore, by selecting elements randomly, it is possible to suppress the occurrence of the first bias.

The policy planning unit 110 determines whether there is a policy for which the allocation of elements to the comparison group has been completed (step S106). For example, if the number of elements in each of group A and group B is greater than a threshold, or if the population has been depleted, it is determined that there is a policy for which the allocation of elements to the comparison group has been completed.

If there are no measures for which the allocation of elements to the comparison group has been completed, the measure planning unit 110 executes the measure loop processing again.

If there is a policy for which the allocation of elements to the comparison group has been completed, the policy planning unit 110 excludes that policy from the selection targets (step S107). Specifically, the policy planning unit 110 deletes the entry for that policy from the list of policies.

The policy planning unit 110 determines whether the allocation of all policy elements has been completed (step S108).

If allocation of all policy elements has not been completed, the policy planning unit 110 returns to step S103 and executes the policy loop processing again.

When all policy elements have been assigned, the policy planning unit 110 terminates the policy implementation plan generation process.

Figures 10A and 10B are flowcharts illustrating an example of the bias evaluation index calculation process executed by the policy evaluation system 100 of Example 1.

After the policy implementation plan generation process is completed, the evaluation index calculation unit 130 of the bias analysis unit 111 starts the process described below. Note that the trigger for executing the process is not limited to this. For example, the policy implementation plan generation process may be executed when an execution instruction is received from the terminal 101.

The evaluation index calculation unit 130 refers to the policy implementation plan management information 123 to identify policies and starts policy loop processing (step S201). Specifically, the evaluation index calculation unit 130 refers to the policy implementation plan management information 123 to generate a list of policies and select one policy from the list of policies.

The evaluation index calculation unit 130 calculates the first bias index for the selected policy (step S202).

For example, the evaluation index calculation unit 130 calculates the first bias index of the policy using an index indicating the variation in the attributes of the targets between group A and group B, and an index indicating the variation in the attributes of the targets' resources between group A and group B.

The evaluation index calculation unit 130 adds an entry to the first bias analysis information 700, sets the identification information of the selected policy in the policy ID 701, and sets the first bias index in the group AB bias 702.

The evaluation index calculation unit 130 starts loop processing of the comparison groups (step S203). Here, it is assumed that the comparison groups are selected in the order of group A and group B. At this time, the evaluation index calculation unit 130 adds an entry to the second bias analysis information 710, sets the identification information of the selected measure in the measure ID 711 of the entry, and sets the type of the selected comparison group in the AB type 712.

For the selected policy, the evaluation index calculation unit 130 extracts a set of targets (target population) that meet the allocation conditions for targets in the comparison group, and extracts a set of resources (resource population) that meet the allocation conditions for resources in the comparison group (step S204).

The evaluation index calculation unit 130 starts loop processing of the target attributes (step S205). Specifically, the evaluation index calculation unit 130 selects one attribute from the group of target attributes.

The evaluation index calculation unit 130 calculates a second bias index for the target attribute (step S206). Specifically, the evaluation index calculation unit 130 calculates the similarity between the attribute distributions of the comparison group and the population as the second bias index. For example, the Jaccard coefficient is calculated as the similarity. At this time, the evaluation index calculation unit 130 sets the similarity in the field corresponding to the selected target attribute in the distribution bias 713 of the entry added in step S203.

The evaluation index calculation unit 130 determines whether the second bias index is equal to or less than a threshold value (step S207). In this embodiment, if the distribution of the attributes of the comparison group is not similar to the distribution of the attributes of the population, i.e., if the second bias index is equal to or less than a threshold value, it is determined that there is a possibility that a second bias has occurred.

If the second bias index is greater than the threshold, the evaluation index calculation unit 130 proceeds to step S209.

If the second bias index is equal to or less than the threshold, the evaluation index calculation unit 130 sets an alert (step S208) and then proceeds to step S209.

In step S209, the evaluation index calculation unit 130 determines whether processing has been completed for all target attributes (step S209).

If processing has not been completed for all target attributes, the evaluation index calculation unit 130 selects a new target attribute and returns to S206.

When processing has been completed for all target attributes, the evaluation index calculation unit 130 starts loop processing of the resource attributes (step S210). Specifically, the evaluation index calculation unit 130 selects one attribute from the group of resource attributes.

The evaluation index calculation unit 130 calculates a second bias index for the resource attribute (step S211). Specifically, the evaluation index calculation unit 130 calculates the similarity between the attribute distributions of the comparison group and the population as the second bias index. For example, the Jaccard coefficient is calculated as the similarity. At this time, the evaluation index calculation unit 130 sets the similarity in the field corresponding to the attribute of the selected resource in the distribution bias 713 of the entry added in step S203.

The evaluation index calculation unit 130 determines whether the second bias index is less than or equal to the threshold value (step S212).

If the second bias index is greater than the threshold, the evaluation index calculation unit 130 proceeds to step S214.

If the second bias index is equal to or less than the threshold, the evaluation index calculation unit 130 sets an alert (step S213) and then proceeds to step S214.

In step S214, the evaluation index calculation unit 130 determines whether processing has been completed for all attributes of the resource (step S214).

If processing has not been completed for all resource attributes, the evaluation index calculation unit 130 selects a new resource attribute and returns to S211.

When processing has been completed for all attributes of the resource, the evaluation index calculation unit 130 determines whether processing for each comparison group has been completed (step S215).

If processing of each comparison group has not been completed, the evaluation index calculation unit 130 selects a new comparison group and returns to step S204.

When processing for each comparison group is complete, the evaluation index calculation unit 130 determines whether processing has been completed for all measures (step S216).

If processing has not been completed for all measures, the evaluation index calculation unit 130 selects a new measure and returns to step S202.

When processing has been completed for all measures, the evaluation index calculation unit 130 ends the bias evaluation index calculation process. The bias analysis unit 111 displays the entries for which an alert has been set on the terminal 101.

In step S204, the evaluation index calculation unit 130 may randomly select a predetermined number of targets from the target population to generate a target set for processing, and may also randomly select a predetermined number of targets from the resource population to generate a resource set for processing. In this case, in step S206, the similarity between the attribute distributions of the comparison group and the processing set is calculated.

According to Example 1, the policy evaluation system 100 can quantitatively evaluate the intensity of the occurrence of the second bias and present the evaluation results. This allows the user to review policy definitions and change settings, etc.

The policy evaluation system 100 of Example 2 analyzes the factors that caused the second bias. Below, Example 2 will be explained, focusing on the differences from Example 1.

The configuration of the computer system in Example 2 is the same as that in Example 1. The policy definition management information 120, resource management information 121, policy target management information 122, and policy implementation plan management information 123 in Example 2 are the same as those in Example 1. The first bias analysis information 700 in the bias analysis result management information 124 in Example 2 is the same as that in Example 1.

In Example 2, the second bias analysis information 710 is different. Figure 11 is a diagram showing an example of the second bias analysis information 710 in Example 2.

An entry in the second bias analysis information 710 includes a bias factor score 714. The bias factor score 714 is a group of fields that store scores indicating the degree of influence of the combination of the target and resource allocation conditions of the measure corresponding to the entry (target measure) and the target and resource allocation conditions of another measure (comparison measure) on the occurrence of the second bias. In this embodiment, the bias factor score 714 stores the degree of overlap between the population of elements that match the allocation conditions of the target measure and the population of elements that match the allocation conditions of the comparison measure as a score, as shown in equation (1).

Figure 12 shows an example of a screen displayed by the terminal 101 in Example 1.

The display unit 141 of the terminal 101 displays a screen 900 for inputting a policy definition.

Screen 1200 includes a group A setting area 1210, a group B setting area 1220, a policy ID setting field 1230, and a registration button 1240.

The group A setting area 1210 includes fields for setting the allocation conditions for resources and targets for group A and the actions for group A. The group B setting area 1220 includes fields for setting the allocation conditions for resources and targets for group B and the actions for group B. The policy ID setting field 1230 is a field for setting identification information for the policy. The register button 1240 is an operation button for registering a policy definition.

In Example 2, before registering the policy definition data, the display unit 141 inquires about the contents of the group A setting area 1210 and the group B setting area 1220 and displays the number of resources and targets occupied. The denominator of the number of occupancies is the number of elements that meet the allocation conditions, and the numerator is the number of elements to be allocated to the comparison group.

Figure 13 is a flowchart illustrating an example of factor analysis processing executed by the policy evaluation system 100 of Example 2. Figure 14 is a diagram illustrating an example of a screen displayed by the terminal 101 of Example 2.

After the policy implementation plan generation process is completed, the factor analysis unit 131 of the bias analysis unit 111 starts the process described below. Note that the trigger for executing the process is not limited to this. For example, the factor analysis process may be executed when an execution instruction is received from the terminal 101.

The factor analysis unit 131 starts policy loop processing (1) (step S301). Specifically, the factor analysis unit 131 references the policy implementation plan management information 123 and selects one policy. In the following description, the selected policy will be referred to as the target policy.

The factor analysis unit 131 starts loop processing (1) of the comparison groups for the target measure (step S302). Here, the comparison groups are selected in the order of group A and group B.

For the target measure, the factor analysis unit 131 extracts a set of targets (target population) that meet the target allocation conditions of the comparison group, and extracts a set of resources (resource population) that meet the resource allocation conditions of the comparison group (step S303).

The factor analysis unit 131 starts policy loop processing (2) (step S304). Specifically, the factor analysis unit 131 references the policy implementation plan management information 123 and selects one policy. Note that the target policy is excluded from the selection targets. In the following explanation, the selected policy will be referred to as the comparison policy.

The factor analysis unit 131 starts loop processing (2) of the comparison groups for the comparison measures (step S305). Here, the comparison groups are selected in the order of group A and group B.

The factor analysis unit 131 extracts a set of targets (target population) that match the target allocation conditions of the comparison group for the comparison measure, and extracts a set of resources (resource population) based on the resource allocation conditions of the comparison group (step S306).

The factor analysis unit 131 calculates the score (target) and score (resource) (step S307). Specifically, the score (target) and score (resource) are calculated using formula (1).

The factor analysis unit 131 references the second bias analysis information 710 and searches for an entry in which the identification information of the target measure is stored in the measure ID 711 and the type of comparison group selected in step S302 is stored in the AB type 712. The factor analysis unit 131 references the bias factor score 714 of the searched entry and sets the calculated score (target) and score (resource) in fields that match the combination of the comparison measure and the type of comparison group selected in step S305.

The factor analysis unit 131 may also calculate the occupancy number and set it in the field.

The factor analysis unit 131 determines whether at least one of the judgment formula (1) and the judgment formula (2) is satisfied (step S308).
Judgment formula (1): Score (target) is greater than or equal to the threshold. Judgment formula (2): Score (resource) is greater than or equal to the threshold.

If at least one of the criteria (1) and (2) is satisfied, it is determined that the combination of the allocation conditions for the target measure and the allocation conditions for the comparison measure is a factor that causes the second bias in the comparison group for the target measure.

If neither judgment formula (1) nor judgment formula (2) is satisfied, the factor analysis unit 131 proceeds to step S310.

If at least one of the criteria (1) and (2) is satisfied, the factor analysis unit 131 sets an alert (step S309) and proceeds to step S310.

In step S310, the factor analysis unit 131 determines whether processing of each comparison group of the comparison measures has been completed (step S310).

If processing of each comparison group of the comparison measures has not been completed, the factor analysis unit 131 selects a new comparison group and returns to step S306.

When processing of each comparison group of comparison measures has been completed, the factor analysis unit 131 determines whether processing has been completed for all measures except the target measure (step S311).

If processing has not been completed for all measures other than the target measure, the factor analysis unit 131 selects a new comparison measure and returns to step S305.

When processing has been completed for all measures except the target measure, the factor analysis unit 131 determines whether processing has been completed for each comparison group of the target measure (step S312).

If processing of each comparison group for the target measure has not been completed, the factor analysis unit 131 selects a new comparison group and returns to step S303.

When processing for each comparison group of the target measures has been completed, the factor analysis unit 131 determines whether processing has been completed for all measures (step S313).

If processing has not been completed for all measures, the factor analysis unit 131 selects a new target measure and returns to step S302.

When processing has been completed for all measures, the factor analysis unit 131 ends the factor analysis processing. The bias analysis unit 111 displays the entry for which the alert has been set on the terminal 101. For example, the terminal 101 displays a screen 1400 such as that shown in FIG. 14. The screen 1400 includes a measure selection field 1410 and alert information 1420 and 1430.

The measure selection field 1410 is a field for selecting a measure for which an alert has been set, and measures for which an alert has been set are displayed in a pull-down menu.

Alert information 1420 is information that indicates the combination of a comparison measure and a comparison group for which an alert has been set, and the score (target). Alert information 1430 is information that indicates the combination of a comparison measure and a comparison group for which an alert has been set, and the score (resource).

According to Example 2, the policy evaluation system 100 can present to the user combinations of allocation conditions for elements that are contributing to the occurrence of the second bias. This allows the user to examine policy definitions and change settings in more detail.

When new policy definition data is registered, the policy evaluation system 100 of Example 3 determines whether a second bias occurs in the relationship between the data and existing policy definition data, and outputs the determination result. Below, Example 3 will be explained, focusing on the differences from Example 1.

The configuration of the computer system in Example 3 is the same as that in Example 1. The policy definition management information 120, resource management information 121, policy target management information 122, and policy implementation plan management information 123 in Example 3 are the same as those in Example 1. The first bias analysis information 700 of the bias analysis result management information 124 in Example 3 is the same as that in Example 1. The second bias analysis information 710 in Example 3 is the same as that in Example 2.

Figure 15 is a diagram showing an example of a screen displayed by the terminal 101 in Example 3. Figure 16 is a flowchart illustrating an example of a policy definition evaluation process executed by the policy evaluation system 100 in Example 3.

The display unit 141 of the terminal 101 displays a screen 800 for inputting a policy definition. The group A setting area 810, group B setting area 820, manual condition setting area 830, policy ID setting field 840, and register button 850 are the same as in Example 1. Specific items in the group A setting area 810, group B setting area 820, and manual condition setting area 830 are omitted. In Example 3, the screen 800 newly includes a bias evaluation result display area 860 and an evaluation button 870.

After making various entries in the Group A setting area 810, Group B setting area 820, manual condition setting area 830, and policy ID setting field 840, the user presses the evaluation button 870. The terminal 101 sends a policy definition evaluation instruction to the policy evaluation system 100. When the policy definition evaluation unit 112 of the policy evaluation system 100 receives the policy definition evaluation instruction, it starts the policy definition evaluation process.

The policy definition evaluation unit 112 instructs the factor analysis unit 131 to execute factor analysis processing (step S401). At this time, the policy definition evaluation unit 112 inputs policy definition data for the new policy to the factor analysis unit 131.

The factor analysis unit 131 executes factor analysis processing with the target measure fixed. In other words, the measure loop processing (1) is not executed. The factor analysis unit 131 does not register data in the second bias analysis information 710, but stores the processing results in the work area. When the factor analysis processing is completed, the factor analysis unit 131 outputs the processing results stored in the work area to the measure definition evaluation unit 112.

The policy definition evaluation unit 112 generates evaluation display information based on the processing results obtained from the factor analysis unit 131 and transmits it to the terminal 101 (step S402). The policy definition evaluation unit 112 then terminates processing.

The terminal 101 displays the bias evaluation result in the bias evaluation result display area 860 based on the evaluation display information.

According to Example 3, the policy evaluation system 100 can support the setting of policy definitions that prevent the occurrence of second bias when setting policy definitions. This allows users to set policy definition data in a way that prevents the occurrence of second bias when registering the data.

The present invention is not limited to the above-described embodiments, but includes various modifications. Furthermore, for example, the above-described embodiments provide detailed descriptions of the configurations in order to clearly explain the present invention, and the present invention is not necessarily limited to those that include all of the described configurations. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with another configuration.

Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be implemented in part or in whole in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor in the computer reads the program code stored on the storage medium. In this case, the program code read from the storage medium itself implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C/C++, Perl, Shell, PHP, Python, and Java (registered trademark).

Furthermore, the software program code that realizes the functions of the embodiments may be distributed via a network and stored on a storage means such as a computer's hard disk or memory, or on a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored on the storage means or storage medium.

In the above examples, the control lines and information lines shown are those considered necessary for explanation, and not all control lines or information lines in the product are necessarily shown. All components may be interconnected.

100 Policy evaluation system 101 Terminal 102 Network 110 Policy planning unit 111 Bias analysis unit 112 Policy definition evaluation unit 120 Policy definition management information 121 Resource management information 122 Policy target management information 123 Policy implementation plan management information 124 Bias analysis result management information 130 Evaluation index calculation unit 131 Factor analysis unit 140 Data reception unit 141 Display unit 200 Computer 201 Processor 202 Main memory device 203 Sub-memory device 204 Network interface 710 Second bias analysis information 800, 900, 1200, 1400 Screen

Claims (12)

A computer system including at least one computer having a processor and a storage device connected to the processor,
retaining policy definition management information that stores policy definition data including allocation conditions for generating a comparison group, which is a set of elements of the policy, used to evaluate the effectiveness of the policy;
The at least one computer
Identifying the comparison group generated based on the policy definition data and a population of the elements that meet the allocation conditions included in the policy definition data;
Calculating the similarity between the distribution of the attributes of the elements included in the comparison group and the distribution of the attributes of the elements included in the population;
A computer system that determines whether or not bias of the elements in the comparison group occurs based on the similarity . 2. The computer system of claim 1,
the policy definition management information stores first policy definition data of a first policy including a first allocation condition and second policy definition data of a second policy including a second allocation condition;
The at least one computer
calculating a score indicating the degree of influence of a combination of the first allocation condition and the second allocation condition on the occurrence of the bias in the comparison group generated based on the first policy definition data, based on a set of elements that match the first allocation condition and a set of elements that match the first allocation condition and the second allocation condition;
A computer system characterized by determining, based on the score, whether the combination of the first allocation condition and the second allocation condition is a factor that causes the bias in the comparison group generated based on the first policy definition data. 2. The computer system of claim 1,
The at least one computer
When a registration request for third policy definition data of a third policy including a third allocation condition is received, the policy definition data stored in the policy definition management information is acquired;
calculating a score indicating the degree of influence of a combination of the third allocation condition and the allocation condition included in the acquired policy definition management information on the occurrence of the bias in the comparison group generated based on the third policy definition data;
A computer system characterized by determining, based on the score, whether the combination of the third allocation condition and the allocation condition contained in the acquired policy definition management information is a factor that causes the bias in the comparison group generated based on the third policy definition data. 2. The computer system of claim 1,
A computer system characterized in that the at least one computer acquires one policy definition data from the policy definition management information and repeatedly executes a process of selecting one element based on the allocation condition contained in the acquired policy definition data, thereby generating the comparison group of policies corresponding to the multiple policy definition data stored in the policy definition management information. A method for evaluating a definition of a policy executed by a computer system, comprising:
The computer system
at least one computer having a processor and a storage device coupled to the processor;
retaining policy definition management information that stores policy definition data including allocation conditions for generating a comparison group, which is a set of elements of the policy, used to evaluate the effectiveness of the policy;
The evaluation method for the definition of the measures is as follows:
a first step in which the at least one computer identifies the comparison group generated based on the policy definition data and a population of the elements that meet the allocation condition included in the policy definition data;
a second step in which the at least one computer determines whether or not bias of the elements in the comparison group has occurred based on a distribution of attributes of the elements included in the comparison group and a distribution of attributes of the elements included in the population,
The second step includes:
a step of calculating a similarity between a distribution of attributes of the elements included in the comparison group and a distribution of attributes of the elements included in the population by the at least one computer;
and a step of determining, by the at least one computer, whether or not the bias has occurred based on the similarity. 6. A method for evaluating a definition of a measure according to claim 5,
the policy definition management information stores first policy definition data of a first policy including a first allocation condition and second policy definition data of a second policy including a second allocation condition;
The evaluation method for the definition of the measures is as follows:
a step in which the at least one computer calculates a score indicating the degree of influence of a combination of the first allocation condition and the second allocation condition on the occurrence of the bias in the comparison group generated based on the first policy definition data, based on a set of elements that match the first allocation condition and a set of elements that match the first allocation condition and the second allocation condition;
A method for evaluating a policy definition, characterized by including a step in which the at least one computer determines, based on the score, whether the combination of the first allocation condition and the second allocation condition is a factor that causes the bias in the comparison group generated based on the first policy definition data. 6. A method for evaluating a definition of a measure according to claim 5,
When the at least one computer receives a request to register third policy definition data of a third policy including a third allocation condition, the at least one computer acquires the policy definition data stored in the policy definition management information;
a step in which the at least one computer calculates a score indicating the degree of influence of a combination of the third allocation condition and the allocation condition included in the acquired policy definition management information on the occurrence of the bias in the comparison group generated based on the third policy definition data;
A method for evaluating a policy definition, characterized by including a step in which the at least one computer determines, based on the score, whether the combination of the third allocation condition and the allocation condition included in the acquired policy definition management information is a factor that causes the bias in the comparison group generated based on the third policy definition data. 6. A method for evaluating a definition of a measure according to claim 5,
A method for evaluating a policy definition, characterized in that it includes a step in which at least one computer acquires one policy definition data from the policy definition management information and repeatedly executes a process of selecting one element based on the allocation condition contained in the acquired policy definition data, thereby generating a comparison group of policies corresponding to multiple policy definition data stored in the policy definition management information. A program for causing a computer to execute a procedure, the program having a processor and a storage device connected to the processor, and holding policy definition management information storing policy definition data including allocation conditions for generating comparison groups, which are sets of elements of the policies, used to evaluate the effectiveness of the policies,
a first step of identifying the comparison group generated based on the policy definition data and a population of the elements that meet the allocation conditions included in the policy definition data;
a second step of determining whether or not bias of the elements in the comparison group has occurred based on a distribution of attributes of the elements in the comparison group and a distribution of attributes of the elements in the population,
The second step comprises:
a step of calculating a similarity between a distribution of attributes of the elements included in the comparison group and a distribution of attributes of the elements included in the population;
and a procedure for determining whether or not the bias occurs based on the degree of similarity. 10. The program according to claim 9,
the policy definition management information stores first policy definition data of a first policy including a first allocation condition and second policy definition data of a second policy including a second allocation condition;
The program
a step of calculating a score indicating the degree of influence of a combination of the first allocation condition and the second allocation condition on the occurrence of the bias in the comparison group generated based on the first policy definition data, based on a set of elements that match the first allocation condition and a set of elements that match the first allocation condition and the second allocation condition;
and a procedure for determining, based on the score, whether the combination of the first allocation condition and the second allocation condition is a factor causing the bias in the comparison group generated based on the first policy definition data. 10. The program according to claim 9,
a step of acquiring the policy definition data stored in the policy definition management information when a registration request for third policy definition data of a third policy including a third allocation condition is received;
a step of calculating a score indicating the degree of influence of a combination of the third allocation condition and the allocation condition included in the acquired policy definition management information on the occurrence of the bias in the comparison group generated based on the third policy definition data;
A program characterized by causing the computer to execute the following steps: based on the score, determine whether the combination of the third allocation condition and the allocation condition included in the acquired policy definition management information is a factor that causes the bias in the comparison group generated based on the third policy definition data. 10. The program according to claim 9,
A program characterized by causing a computer to execute a procedure for generating a comparison group of measures corresponding to multiple policy definition data stored in the policy definition management information by repeatedly executing a process of obtaining one policy definition data from the policy definition management information and selecting one element based on the allocation condition contained in the obtained policy definition data.