JP7522697B2 - Test case verification device and test case verification method - Google Patents

Description

The present invention relates to a test case verification device and a test case verification method that verify the validity of test cases that define test content in a simulator.

In recent years, to improve the efficiency of development of automobiles and the like, simulators such as Hardware In the Loop Simulation (HILS) are being used, which quantifies the engine functions and behavior of vehicles to enable simulations that mimic the actual equipment, as described in JP 2011-118841 A (Patent Document 1). In simulators, tests are performed according to test cases in which the values of input parameters that define the test contents, their changes, and the expected values of output parameters are written in functional form. Test cases are created by referring to a test requirements definition document that summarizes the test requirements, and by designers manually converting this into functional form.

JP 2011-118841 A

However, because designers and others create test cases by referring to the test requirements definition document, there is room for human error in the test case creation process, and there is a risk that the test requirements will not necessarily be reflected in the test cases. In addition, because test cases describe test procedures, which include time information, and judgment criteria, which do not include time information, in functional form, it is difficult to grasp the changes over time in the output parameters for the judgment criteria, and it is also difficult to judge whether the test case is good or bad.

The present invention aims to provide a test case verification device and a test case verification method that can easily verify the pass/fail of test cases created from a test requirements definition document in testing using a simulator.

The test case verification device reads a test requirements definition document that defines a graph showing the change over time of an output parameter output from a simulator, a test procedure in which the values of input parameters to be input to the simulator and their retention times are described in functional form, and a test case including a judgment criterion in which the values of output parameters output from the simulator are described in functional form. The test case verification device then executes the following processes: extracting time information at which the values of the input parameters change from the test procedure of the test case and applying it to the judgment criterion to create an expected value graph showing the change over time of the output parameters; performing image recognition on the graph in the test requirements definition document to create an assumed value graph that can be compared with the expected value graph; and comparing the expected value graph with the assumed value graph to verify the pass/fail of the test case.

In the test case verification method, a computer reads a test requirements definition document that defines a graph showing the change over time of an output parameter output from a simulator, a test procedure in which the values of input parameters to be input to the simulator and their retention times are described in functional form, and a test case including a judgment criterion in which the values of output parameters output from the simulator are described in functional form. The computer then executes the steps of: extracting time information at which the values of the input parameters change from the test procedure of the test case and applying it to the judgment criterion to create an expected value graph showing the change over time of the output parameters; performing image recognition on the graph in the test requirements definition document to create an assumed value graph that can be compared with the expected value graph; and verifying the pass/fail of the test case by comparing the expected value graph with the assumed value graph.

According to the present invention, in testing using a simulator, it is possible to easily verify the validity of test cases created from a test requirements specification document.

FIG. 1 is a schematic diagram illustrating an example of a system for testing an electronic control device. FIG. 1 is an explanatory diagram illustrating an example of a test requirement definition document. FIG. 11 is an explanatory diagram illustrating an example of a test case. FIG. 11 is an explanatory diagram showing an example of a graph created from a test procedure. FIG. 13 is an explanatory diagram of the function “Check_Values” that can be set as the judgment criteria. FIG. 11 is an explanatory diagram of a function “Check_Accuracy” that can be set as a judgment criterion. FIG. 11 is an explanatory diagram of the function “Check_Response_Time” that can be set as the judgment criterion. FIG. 13 is an explanatory diagram of an expected value graph created by linking a plurality of actions. FIG. 1 is a schematic diagram illustrating an example of a test case authentication device that authenticates a test case. 13 is a flowchart illustrating an example of a verification process. 13 is a flowchart illustrating an example of a verification process. FIG. 11 is an explanatory diagram of symbols that define the change direction of an output parameter. FIG. 11 is an explanatory diagram showing an example in which the change direction of an output parameter is represented by a symbol. FIG. 11 is an explanatory diagram showing an example of an expected output graph symbolized taking into account the length of an output parameter. FIG. 11 is an explanatory diagram of a method for defining the length of an output parameter. FIG. 13 is an explanatory diagram of a method for comparing an expected value graph with a predicted value graph. FIG. 11 is an explanatory diagram showing an example of a test result. FIG. 11 is an explanatory diagram showing an example of a test result. FIG. 11 is an explanatory diagram showing an example of a test result. FIG. 11 is an explanatory diagram illustrating an example of a test report.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Fig. 1 shows an example of a system for testing the functions of an electronic control unit 100 mounted on a vehicle, etc. The system shown in Fig. 1 is not limited to the electronic control unit 100 mounted on a vehicle, etc., but can also test the functions of the electronic control unit 100 mounted on, for example, industrial equipment, etc.

The system for testing the functions of the electronic control unit 100 includes a HILS 200 implemented by a computer system (not shown) such as a personal computer. The computer system implements HILS 200, which is an example of a simulator that quantifies the engine functions and behavior of a vehicle, which is an example of an object controlled by the electronic control unit 100, and enables a simulation that mimics the actual machine, by executing an application program stored in a storage device such as a hard disk drive or SSD (Solid State Drive).

HILS200 tests the functions of the electronic control unit 100 according to test cases 400 in which the test contents of the electronic control unit 100 are described in functional form, which are created by a designer or the like from a test requirements definition document 300 in which the test requirements of the electronic control unit 100 are defined. Specifically, HILS200 inputs a control signal from the electronic control unit 100, executes a simulation corresponding to the control signal, and outputs the resulting system state to the electronic control unit 100. Then, HILS200 sequentially exchanges control signals and system states with the electronic control unit 100, and tests whether the electronic control unit 100 can perform the required functions. Note that HILS200 is a well-known technology to those skilled in the art, so a detailed description thereof will be omitted.

As shown in FIG. 2, the test requirements definition document 300 defines the values and changes of the input parameters input to the HILS 200, as well as a graph showing the changes over time of the output parameters output from the HILS 200.

In the illustrated test requirements definition document 300, the input parameters are defined as "HVDC Voltage" set to 800 [V], "LV switch Voltage" set to 12 [V], and "Wakeup request" set to "high", and the "Mode command" is defined to change in a step response to "Idol mode", "Torque control mode", and "Three phase short mode".

In addition, in the illustrated test requirements definition document 300, it is defined that, as output parameters, the "System state" changes in a step response to "Idle mode", "Torque control mode" and "Three phase short mode" with a predetermined delay from the change in the "Mode command", and the "State number" changes in a step response to "Idol mode number", "Torque control mode number" and "Three phase short mode number" with a predetermined response time from the change in the "System state". Note that the illustrated test requirements definition document 300 is merely an example for the purpose of explanation, and should not be understood as necessarily being like this.

As shown in FIG. 3, the test case 400 includes a test procedure 420 in which the values of input parameters to be input to the HILS 200 and their retention times are described in functional form, and Expected Results 440 in which the values of output parameters output from the HILS 200 are described in functional form.

Test procedure 420 is composed of at least one record that associates, for each action related to one input parameter, a sequence number indicating the order of processing, a function (Function) specifying what to do, and a variable name (Variable_Name), its value (Value), and unit (Unit) as arguments to the function. Functions in test procedure 420 include "Set", which sets a value to a variable, "Ramp", which changes the variable in a ramp response depending on the value, and "Wait", which holds the current state for the value [units].

The judgment criteria 440 consists of at least one record that is associated with each action related to one input parameter, a sequence number indicating the order of processing, a function (Function) specifying what to do, and a variable name (Variable_Name), its value (Value) and unit (Unit) as arguments to the function. Functions of the judgment criteria 440 include "Monitor" which monitors variables, "Check_Values" which checks that variables are constant, "Check_Accuracy" which checks the accuracy of a variable that responds in steps, and "Check_Response_Time" which checks the response time of a variable that responds in steps.

In the illustrated test case 400, the test procedure 420 describes setting the variable "INV_Mode" to "0" and maintaining that state for "3 seconds." The judgment criterion 440 describes monitoring the variable "vBat_Voltage_Limit," then monitoring the variable "flgninition_Switch," and then checking whether the variable "flgninition_Switch" becomes "1." Note that the test case 400 shown in FIG. 3 is merely an example for the purpose of explanation, and should not be understood as necessarily being like this.

Here, we will explain the test case 400 using a specific example to make it easier to understand.

In test procedure 420 of test case 400, as shown in FIG. 4, consider the following: "Action 3" sets the variable "Mot_Tmp" to "-60°C" and holds that state for "5 seconds," and "Action 4" sets the variable "Mot_Tmp" to "-50°C" and holds that state for "5 seconds."

If a graph is created with the horizontal axis representing elapsed time and the vertical axis representing the variable value, for "action 3", the graph will show that the variable "Mot_Tmp" is set to "-60 [°C]" at time "0 [seconds]" and maintained in that state for "5 [seconds]", and for "action 4", the graph will show that the variable "Mot_Tmp" is set to "-50 [°C]" at time "0 [seconds]" and maintained in that state for "5 [seconds]". And because "action 3" and "action 4" are executed consecutively, if the graph for "action 4" is linked to the graph for "action 3", the graph will show that the variable "Mot_Tmp" is set to "-60 [°C]" at time "0 [seconds]" and maintained in that state for "5 [seconds]", and then at time "5 [seconds]", the variable "Mot_Tmp" changes from "-60 [°C]" to "-50 [°C]" and maintained in that state for "5 [seconds]". By looking at this graph, you can see that it contains time information such as the time to set the variable "Mot_Tmp", the time to change it, and the time to end the hold.

In the judgment criteria 440 of the test case 400, the processing content related to the judgment differs depending on the functions "Check_Values", "Check_Accuracy", and "Check_Response_Time". Each function is explained below.

<<Function “Check_Values”>>
In the test procedure 420 of the test case 400, as shown in FIG. 5, if it is described that the variable "Mot_Tmp" is set to "50[°C]" and the state is maintained for "5[seconds]", the graph created from this will show that the variable "Mot_Tmp" is set to "50[°C]" at time "0[seconds]" and the state is maintained for "5[seconds]", that is, the state continues until time "10[seconds]". In this way, if the input parameters take constant values that do not change over time, it can be assumed that the output parameters output from the HILS 200 also take constant values. In this case, in the judgment criteria 440 of the test case 400, for the function that specifies what to do, it is possible to adopt "Check_Values" that checks that the value of the variable is a constant value. The function "Check_Values" requires at least the variable name, its value, and unit as its arguments.

In one example of the judgment criteria 440 shown in FIG. 5, it is described that "Check_Values" is adopted as the function, and it is confirmed that the variable "vMotor_Temperature" is "50 [°C]". However, although the function "Check_Values" defines the value "50 [°C]" that the variable "vMotor_Temperature", which is an output parameter, can take, there is no time information about when this value will be established, and there is a problem in that it is difficult to grasp the change in the variable "vMotor_Temperature" over time.

Then, from the graph created from the test procedure 420 of the test case 400, the time "0 seconds" when the variable "Mot_Tmp" is set to "50°C" and the time "5 seconds" when this state is maintained are extracted as time information. The times thus extracted are then applied to the judgment criterion 440 to create an expected value graph showing that the variable "vMotor_Temperature", which is an output parameter, becomes "50°C" at time "0 seconds" and maintains this state for "5 seconds". Here, the expected value graph is a graph that represents the change over time in the output parameter that is expected to be output by the HILS 200 when the HILS 200 executes a simulation according to the test procedure 420 of the test case 400 (same below).

<<Function “Check_Accuracy”>>
6, if it is described that the variable "Mot_Tmp" is set to "50[° C.]" and the state is held for "2 [seconds]", then the variable "Mot_Tmp" is set to "60[° C.]" and the state is held for "3 [seconds]", the graph created from this shows that the variable "Mot_Tmp" is set to "50[° C.]" at time "0 [seconds]" and the state is held for "2 [seconds]", then the variable "Mot_Tmp" changes from "50[° C.]" to "60[° C.]" at time "2 [seconds]" and the state is held for "3 [seconds]", that is, the state is sustained for a time "5 [seconds]". In this way, when the input parameters change in a step response, it can be assumed that the output parameters output from the HILS 200 also change in a step response. In this case, "Check_Accuracy" that checks the accuracy of the step-response variable can be adopted as a function that specifies what to do in the judgment criteria 440 of the test case 400. The function "Check_Accuracy" requires, as its arguments, at least the variable name, two values before and after the change, the unit, the allowable range of each value, and the condition of the value after the change.

In an example of the judgment criteria 440 shown in FIG. 6, the function "Check_Accuracy" is adopted, and it is described that it is confirmed that the variable "vMotor_Temperature" changes from "50±2°C" to "60±5°C" in response to a change in the variable "Mot_Tmp" which is an input parameter, and that the state in which the variable "vMotor_Temperature" becomes "60±5°C" after "100 milliseconds" have elapsed since the change in the variable "vMotor_Temperature" continues for at least "2000 milliseconds." However, although the function "Check_Accuracy" defines two values before and after the change in the variable "vMotor_Temperature" which is an output parameter, the allowable range of each value, and the condition of the value after the change, there is no time information about when this is established, and it is difficult to grasp the change state of the variable "vMotor_Temperature" over time.

Then, from the graph created from the test procedure 420 of the test case 400, the time "0 [seconds]" when the variable "Mot_Tmp" is set to "50 [°C]", the time "2 [seconds]" when the variable "Mot_Tmp" changes from "50 [°C]" to "60 [°C]", and the time "3 [seconds]" when the variable "Mot_Tmp" is maintained at "60 [°C]" are extracted as time information. Then, the times extracted in this way are applied to the judgment criterion 440, and an expected value graph is created which shows that after the variable "vMotor_Temperature" becomes "50±2 [°C]" at time "0 [seconds]" and is maintained in this state for "2 [seconds]", the variable "vMotor_Temperature" changes from "50±2 [°C]" to "60±5 [°C]", and after "0.1 [seconds]" has elapsed from that point, this state is maintained for at least "2 [seconds]".

<<Function “Check_Response_Time”>>
7, if it is described that the variable "Mot_Tmp" is set to "20[° C.]" and the state is held for "1 [second]", then the variable "Mot_Tmp" is set to "100[° C.]" and the state is held for "4 [seconds]", the graph created from this shows that the variable "Mot_Tmp" is set to "20[° C.]" at time "0 [seconds]" and the state is held for "1 [second]", then the variable "Mot_Tmp" changes from "20[° C.]" to "100[° C.]" at time "1 [second]" and the state is held for "4 [seconds]", that is, the state continues until time "5 [seconds]". In this way, when the input parameters change in a step response, it can be assumed that the output parameters output from the HILS 200 also change in a step response. In this case, "Check_Response_Time" that checks the response time of a variable that responds in steps can be adopted as a function that specifies what to do in the judgment criteria 440 of the test case 400. The function "Check_Response_Time" requires at least the variable name, two values before and after the change, a unit, and the time required for the change (Timing) as its arguments.

In one example of the judgment criteria 440 shown in FIG. 7, "Check_Response_Time" is adopted as a function, and it is described that it is confirmed that the variable "vMotor_Temperature" changes from "20°C" to "100°C" in response to a change in the variable "Mot_Tmp", which is an input parameter, and that the time required for this change is "120 milliseconds". However, although the function "Check_Response_Time" defines two values before and after the change in the variable "vMotor_Temperature", which is an output parameter, and the time required for this change, there is no time information about when this is established, making it difficult to grasp the change state of the variable "vMotor_Temperature" over time.

Then, from the graph created from the test procedure 420 of the test case 400, the time "0 [seconds]" when the variable "Mot_Tmp" is set to "20 [°C]", the time "1 [second]" when the variable "Mot_Tmp" changes from "20 [°C]" to "100 [°C]", and the time "4 [seconds]" when the variable "Mot_Tmp" is maintained at "100 [°C]" are extracted as time information. The times thus extracted are then applied to the judgment criterion 440, and an expected value graph is created which shows that after the variable "vMotor_Temperature" becomes "20 [°C]" at time "0 [seconds]" and maintains this state for "1 [second]", the variable "vMotor_Temperature" changes from "20 [°C]" to "100 [°C]" over time "0.12 [seconds]", and the state is maintained for at least "3.88 [seconds]".

At least one of the three functions mentioned above, "Check_Values", "Check_Accuracy" and "Check_Response_Time", can be set as the judgment criteria 440 of the test case 400. In this case, the priority order should be "Check_Response_Time" > "Check_Accuracy" > "Check_Values".

When multiple actions are defined in test case 400, the expected value graphs created as described above can be linked in sequence to create an expected value graph that shows the expected values of the output parameters obtained as a result of a series of tests, as shown in Figure 8. The expected value graph shown in the figure shows one continuous expected value graph obtained for "action 0" to "action 3," with the response time and accuracy set.

However, since the test cases 400 are manually created by designers or the like based on the test requirements definition document 300, there is a possibility that human error may occur in the process of creating the test cases 400, and the test requirements may not necessarily be reflected in the test cases 400. If the test requirements are not reflected in the test cases 400, the HILS 200 cannot test the electronic control device 100 according to the test requirements, and there is a risk that, for example, testing of a certain function may be inappropriate.

As shown in FIG. 9, a test case verification device 500 capable of reading the test requirements definition document 300 and the test cases 400 is used to verify the pass/fail of the test cases 400 created from the test requirements definition document 300. The test case verification device 500 is implemented by a computer system executing an application program, and may be incorporated in the HILS 200 or may be separate from the HILS 200.

Figures 10 and 11 show an example of a verification process executed by the test case verification device 500. Here, as a prerequisite for executing the verification process, it is assumed that a test requirements definition document 300 and multiple test cases 400 created from it are stored in storage.

In step 10 (abbreviated as "S10" in FIG. 10, and the same applies below), the test case verification device 500 displays a selection screen on the display device of the computer system for selecting a test case 400 to be verified from among the multiple test cases 400 stored in the storage. The selection screen can be any screen, such as a screen for interactively selecting a test case 400 using a pointing device or the like, or a screen for directly inputting the identifier of the test case 400 to be verified.

In step 11, the test case verification device 500 determines whether or not a test case 400 to be verified has been selected from among the multiple test cases 400. If the test case verification device 500 determines that a test case 400 to be verified has been selected (Yes), the process proceeds to step 12. On the other hand, if the test case verification device 500 determines that a test case 400 to be verified has not been selected (No), the device waits until a test case 400 to be verified is selected.

In step 12, the test case verification device 500 creates an expected value graph from the test procedure 420 and the judgment criteria 440 of the test case 400 to be verified. Specifically, the test case verification device 500 creates a graph showing input parameters that change over time from the test procedure 420 of the test case 400, and extracts time information from the graph. Then, the test case verification device 500 applies the extracted time information to the judgment criteria 440 of the test case 400 to create at least one expected value graph showing output parameters that change over time.

In step 13, the test case verification device 500 determines whether or not there are multiple expected value graphs created in step 12. If the test case verification device 500 determines that there are multiple expected value graphs (Yes), it proceeds to step 14. On the other hand, if the test case verification device 500 determines that there is only one expected value graph (No), it proceeds to step 16.

In step 14, the test case verification device 500 displays a selection screen on the display device of the computer system for selecting an expectation graph to be verified from the multiple expectation graphs created in step 12. The selection screen can be any screen, such as a screen for interactively selecting an expectation graph using a pointing device or a screen for directly inputting the identifier of the expectation graph to be verified.

In step 15, the test case verification device 500 determines whether or not an expected value graph to be verified has been selected from among the multiple expected value graphs. If the test case verification device 500 determines that an expected value graph to be verified has been selected (Yes), the process proceeds to step 16. On the other hand, if the test case verification device 500 determines that an expected value graph to be verified has not been selected (No), the test case verification device 500 waits until an expected value graph to be verified is selected.

In step 16, the test case verification device 500 symbolizes the expected value graph created in step 12 or the expected value graphs selected in steps 13 and 14, as described in detail below.

<<Symbolization of graphs>>
As symbols representing the direction of change of the output parameter that changes with time, as shown in Fig. 12, "U" representing an upward change, "UR" representing a change diagonally to the upper right due to a ramp response, "R" representing a change to the right, "DR" representing a change diagonally to the lower right due to a ramp response, and "D" representing a downward change are used. To explain this with a specific example, as shown in Fig. 13, when the output parameter changes to the right, upward, right, lower right, and right in this order with time, the direction of change can be represented by the letters "R", "U", "R", "DR", and "R", respectively. Also, when the output parameter is held in a state of "0" for "5 [seconds]" as shown in Fig. 14, and then the value changes from "0" to "1" and is held in that state for "3 [seconds]", the expected value graph can be symbolized as "R5U1R3". At this time, the reference lengths in the vertical and horizontal directions for the graph representing the output parameters are preset, and the expected value graph can be symbolized using these reference lengths, as shown in Fig. 15. Note that the example shown in Fig. 15 shows that when the expected value graph is symbolized, the character string "R6U1R4U1R6" is obtained.

In step 17, the test case verification device 500 extracts an expected value graph associated with the expected value graph to be verified from the test requirements definition document 300. Here, it is assumed that at least one expected value graph has been created in the test requirements definition document 300 using, for example, a drawing tool.

In step 18, the test case verification device 500 symbolizes the expected value graph by image recognition. Note that the symbolization of the expected value graph is performed through a process similar to that of the expected value graph, so its explanation is omitted. If necessary, please refer to the explanation of the expected value graph symbolization.

In step 19, the test case verification device 500 compares the expected value graph symbolized in step 16 with the assumed value graph symbolized in step 18. Specifically, as shown in FIG. 16, the test case verification device 500 compares, for example, the character string "R7U5R4" obtained by symbolizing the expected value graph with the character string "R6U2R4" obtained by symbolizing the assumed value graph, and determines whether the symbols indicating the change direction of the output parameter match. In addition to the change direction of the output parameter, the test case verification device 500 also determines whether the length of the output parameter is within a preset allowable range. In the illustrated example, if the allowable range in the up-down direction is "3" and the allowable range in the left-right direction is "2", "R7" and "6", "U5" and "U2", and "R4" and "R4" are compared, respectively, and it is determined that they match because they are within the allowable range.

In step 20, the test case verification device 500 determines whether the expected value graph and the assumed value graph match, based on the comparison result in step 19. If the test case verification device 500 determines that the expected value graph and the assumed value graph match (Yes), it proceeds to step 21. On the other hand, if the test case verification device 500 determines that the expected value graph and the assumed value graph do not match (No), it proceeds to step 22.

In step 21, the test case verification device 500 determines that the test case 400 to be verified is valid because the expected value graph and the assumed value graph match. The test case verification device 500 then proceeds to step 23.

In step 22, the test case verification device 500 determines that the test case 400 to be verified is invalid because the expected value graph and the assumed value graph do not match. The test case verification device 500 then proceeds to step 23.

In step 23, the test case verification device 500 creates test results as shown in Figs. 17 to 19. Specifically, the test case verification device 500 creates data that visualizes the verification results and the contents defined in the test case 400 for each action of the test case 400. Here, the test results of "Action 0" and "Action 1" shown in Figs. 17 and 18 indicate that the verification by "Check_Accuracy" of the judgment criterion 440 for the output parameter "vSpeedMotor" is OK, and the test result of "Action 2" shown in Fig. 19 indicates that the verification by "Check_Accuracy" of the judgment criterion 440 for the output parameter "vSpeedMotor" is NG. Note that the test results are not limited to those shown in Figs. 17 to 19, and can be any that can identify the verification target and its verification result.

In step 24, the test case verification device 500 outputs and displays the test results created in step 23 on the display device of the computer system. Therefore, by checking the test results displayed on the display device, the designer or the like can easily determine whether there is a problem with the test case 400 and, if there is a problem, what the problem is. If there is a problem with the test case 400, the designer or the like can easily correct the problem with the test case 400 by focusing on the part that is causing the problem.

In step 25, the test case verification device 500 displays a screen for prompting the user to specify whether or not to continue the verification of the test case 400, and determines whether or not to end the verification process depending on the result of the specification. If a specification to end the verification process has been made (Yes), the test case verification device 500 ends the verification process. On the other hand, if a specification to continue the verification process has not been made (No), the test case verification device 500 returns the process to step 10 to continue the verification of other test cases 400. The test case verification device 500 may also display a screen for prompting the user to specify whether or not to continue the verification of other expected value graphs.

According to this verification process, the test case verification device 500 extracts time information at which the input parameters change from the test procedure 420 of the test case 400 and applies it to the judgment criterion 440 to create an expected value graph that shows the change in the output parameters over time, and generates a character string that symbolizes this. The test case verification device 500 also performs image recognition on the graph of the test requirements definition document 300 to create an assumed value graph that can be compared with the expected value graph, and generates a character string that symbolizes this. The test case verification device 500 then compares the two character strings thus generated and determines whether the expected value graph and the assumed value graph match, in other words, whether the expected value graph reflects the contents of the test requirements definition document 300. Therefore, in a test using a simulator such as the HILS 200, the pass/fail of the test case 400 created from the test requirements definition document 300 can be easily verified.

At this time, the test case verification device 500 compares the two character strings, taking into account the tolerances set for the timing of change in the value of the output parameter and the retention time. Therefore, even if there are some differences between the two character strings due to, for example, the precision with which the expected value graph and the assumed value graph are created, and the precision with which they are symbolized to generate the character string, it can be determined that the two character strings match.

Instead of or in addition to creating and outputting the test results, the test case verification device 500 may create and output a test report 600 including a report title, use case/test case name, test result summary, test overview, ID list, expected value graph, test purpose, test environment list, overall test graph, and test results, as shown in FIG. 20. Note that the test report 600 shown in FIG. 20 is merely an example for the purpose of explanation, and may include any items necessary for a test report.

A person skilled in the art will easily understand that new embodiments can be created by omitting parts of the technical ideas of the various embodiments described above, combining parts of them as appropriate, or replacing parts of them with well-known technology.

As one example, the test requirements definition document 300 is not limited to one created using a drawing tool, but may be drawn on paper, for example. In this case, the test case verification device 500 may read the paper using, for example, a scanner or digital camera, and apply well-known technology to at least recognize the graph, and then create an expected value graph from this.

200 HILS (Simulator)
300 Test requirements definition document 400 Test case 420 Test procedure 440 Judgment criteria 500 Test case verification device 600 Test report

Claims (6)

A test case verification device that reads a test requirement definition document in which a graph showing a change over time of an output parameter output from a simulator is defined, a test procedure in which values of input parameters to be input to the simulator and their retention times are described in functional form, and a test case including a judgment criterion in which values of output parameters output from the simulator are described in functional form, and verifies pass/fail of the test case, comprising:
a process of extracting time information on the change in the value of the input parameter from the test procedure of the test case, applying the time information to the judgment criterion, and creating an expected value graph showing the change in the output parameter over time;
A process of performing image recognition on the graph of the test requirements definition document and creating an assumed value graph that can be compared with the expected value graph;
a process of comparing the expected value graph with the assumed value graph to verify whether the test case is correct;
A test case verifier that executes the test case verifier. The process of verifying the pass/fail of the test case includes creating character strings for the expected value graph and the assumed value graph, the character strings representing the change direction of the value of the output parameter and the retention time thereof using predetermined symbols, and comparing the two character strings to determine whether they match.
2. The test case verification apparatus according to claim 1. the process of verifying whether the test case is acceptable or not comprises comparing the two character strings to determine whether they match, taking into consideration allowable ranges set for the timing at which the value of the output parameter changes and the duration for which the output parameter is held.
3. The test case verification apparatus according to claim 2. outputting a verification result of the test case;
4. The test case verification device according to claim 1. outputting a test report including the contents of the test requirements definition document, the expected value graph, and the results of the test executed by the simulator in accordance with the test cases;
5. The test case verification device according to claim 1. A test case verification method in which a computer reads a test requirements definition document in which a graph showing a change over time of an output parameter output from a simulator is defined, a test procedure in which values of input parameters to be input to the simulator and their retention times are described in functional form, and a test case including a judgment criterion in which values of output parameters output from the simulator are described in functional form, and verifies pass/fail of the test case, comprising:
The computer,
extracting time information on the change in the value of the input parameter from the test procedure of the test case, and applying the time information to the judgment criterion to generate an expected value graph showing the change in the output parameter over time;
A step of performing image recognition on a graph of the test requirements definition document and creating an assumed value graph that can be compared with the expected value graph;
A step of comparing the expected value graph with the assumed value graph to verify whether the test case is correct or not;
Run test case verification method.