JP7755247B2 - Program, method, information processing device, and system - Google Patents

Description

This disclosure relates to a program, a method, an information processing device, and a system.

To ensure the quality of software, its functionality is tested before it is released. Software functionality is generally tested based on test specifications, which describe the test method and the conditions for determining pass/fail for each test case.

When it comes to software testing, creating efficient and accurate test plans is important for advancing a project. In Patent Document 1, the amount of work required for unexecuted test items is predicted based on the actual amount of work performed in the past for identical or similar tests.

Patent Publication No. 2020-98413

In software development, the order in which tests are performed is an important factor in ensuring that the testing process in a project proceeds as planned. For example, if there are tests that are likely to cause defects, the longer those tests are performed, the greater the risk of degradation. For this reason, performing tests that are likely to cause defects early in the testing process will allow testing in the testing process to proceed more efficiently.

The purpose of this disclosure is to conduct tests efficiently.

To solve the above problem, the program disclosed herein is a program to be executed by a computer having a processor and memory. The program causes the processor to execute the following steps: storing information about tests performed in a testing process; selecting a test case to be tested with priority from among test cases that have been tested based on the information about the test; extracting test cases similar to the selected test case from untested test cases in the testing process; and setting the priority of testing the extracted test case higher than that of other untested test cases.

This disclosure allows tests to be conducted efficiently.

1 is a diagram showing the overall configuration of a system according to an embodiment of the present invention; FIG. 2 is a block diagram showing the configuration of a terminal device included in the system of the present embodiment. FIG. 2 is a block diagram showing the functional configuration of a server included in the system of the present embodiment. FIG. 10 is a diagram illustrating the data structure of a test case DB stored in the server. FIG. 10 is a diagram illustrating the operation of the system when a test is administered by a tester. FIG. 10 is a schematic diagram illustrating an example of a project list displayed on a display. FIG. 10 is a schematic diagram illustrating an example of a list of test suites displayed on a display. FIG. 10 is a schematic diagram illustrating an example of a list of test case data displayed on a display. FIG. 10 is a schematic diagram illustrating an example of a ticket input form displayed on a display. FIG. 10 is a schematic diagram showing an example of the contents of a ticket displayed on a display. FIG. 1 is a diagram illustrating the operation of the system when software is modified by a developer and the modified software is tested by a tester. 10 is a flowchart illustrating an example of the operation of a server when setting priorities set in test suites. 10 is a flowchart showing a specific example of the operation of an analysis unit. FIG. 10 is a schematic diagram illustrating an example of a list of test suites displayed on a display. FIG. 10 is a schematic diagram illustrating an example of a list of test case data displayed on a display. FIG. 10 is a schematic diagram illustrating an example of a list of test suites displayed on a display. FIG. 10 is a schematic diagram illustrating an example of a list of test case data displayed on a display. 10 is a flowchart illustrating an example of the operation of a server when predicting test trends. FIG. 10 is a schematic diagram illustrating an example of a graph showing test trends displayed on a display.

Embodiments will be described below with reference to the drawings. Note that the embodiments described below do not unduly limit the content of the present disclosure as set forth in the claims. Furthermore, not all of the configurations described in the embodiments are necessarily essential components of the present disclosure.

<Overview>
In this embodiment, system 1 analyzes test cases that have already been executed during the testing process of a specific project and extracts test cases that should be prioritized from among untested test cases. System 1 then prompts the tester to prioritize testing of the extracted test cases. In response to requests from testers, developers, project managers, etc., system 1 then presents the requester with predicted test trends resulting from changes to the test execution order.

In this embodiment, test cases are set based on a test design document. A test design document is data in which tests are designed to test the functions of a test target. A test target represents the object of testing, such as hardware or software developed in a specific project. Test targets include, for example, prototypes, intermediate products, and final products.

<1 Overall structure>
Fig. 1 is a diagram showing the overall configuration of a system 1 according to the present embodiment. As shown in Fig. 1, the system 1 includes a terminal device 10 and a server 20. The terminal device 10 and the server 20 are connected to each other via a network 80 using a wired or wireless communication standard so as to be able to communicate with each other. The network 80 is realized by, for example, the Internet and/or a communication network provided by a telecommunications carrier. In the illustrated example, the system 1 includes a plurality of terminal devices 10.

Note that while Figure 1 shows a case where server 20 is a single computer, server 20 may also be realized by combining multiple computers.

Terminal device 10 is a terminal used by a user who uses the services provided by server 20. For example, terminal device 10-1 is a terminal used by a tester who conducts tests. Terminal device 10-2 is a terminal used by a developer who develops software. Although there are terminal devices 10-1 and 10-2, the structures of the devices themselves are almost the same, so below they will be described as terminal device 10.

The terminal device 10 may be realized, for example, as a desktop personal computer (PC), a laptop PC, a head-mounted display, or the like. The terminal device 10 may also be a portable computer such as a smartphone or a tablet terminal.

As shown in FIG. 1, the terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, storage 16, and a processor 19. The communication IF 12, the input device 13, the output device 14, the memory 15, the storage 16, and the processor 19 are connected to each other so that they can communicate with each other, for example, via a bus.

The communication IF 12 is an interface for sending and receiving signals so that the terminal device 10 can communicate with external devices. The input device 13 is an input device for accepting input operations from users (employees). The input device 13 includes, for example, a touch panel, a touchpad, a pointing device such as a mouse, a keyboard, etc. The output device 14 is an output device for presenting information to the user. The output device 14 includes, for example, a display, a speaker, etc.

Memory 15 is used to temporarily store programs and data processed by programs, etc., and is realized by volatile memory such as DRAM (Dynamic Random Access Memory). Storage 16 is a storage device for saving data, and is realized by non-volatile memory such as flash memory or HDD (Hard Disc Drive). Processor 19 is hardware for executing instruction sets written in programs, and is made up of an arithmetic unit, registers, peripheral circuits, etc.

Server 20 is a device that manages the implementation status and results of tests specified based on the test design document. Server 20 is, for example, a computer connected to network 80.

As shown in FIG. 1, the server 20 includes a communication IF 22, an input/output IF 23, a memory 25, a storage 26, and a processor 29. The communication IF 22, the input/output IF 23, the memory 25, the storage 26, and the processor 29 are communicatively connected to each other, for example, via a bus.

The communication IF 22 is an interface for sending and receiving signals so that the server 20 can communicate with external devices. The input/output IF 23 functions as an interface with an input device for accepting input operations from the user and an output device for presenting information to the user. The memory 25 is used to temporarily store programs and data processed by programs, etc., and is realized by volatile memory such as DRAM.

Storage 26 is a storage device for saving data, and is realized by non-volatile memory such as flash memory or HDD. Storage 26 does not necessarily have to be realized by a single circuit. Storage 26 may be realized by, for example, multiple memory circuits. Processor 29 is hardware for executing a set of instructions written in a program, and is made up of an arithmetic unit, registers, peripheral circuits, etc.

<1.1 Configuration of terminal device>
2 is a block diagram showing the configuration of the terminal device 10 included in the system 1 of this embodiment. As shown in FIG. 2, the terminal device 10 includes a communication unit 121, an input device 13, an output device 14, a storage unit 170, and a control unit 180.

The communication unit 121 performs processing to enable the terminal device 10 to communicate with other devices. The communication unit 121 performs transmission processing on signals generated by the control unit 180 and transmits them to the outside (e.g., server 20). The communication unit 121 performs reception processing on signals received from the outside and outputs them to the control unit 180.

The input device 13 is a device through which the user of the terminal device 10 inputs instructions. The input device 13 is realized, for example, by a mouse 131, a keyboard 132, and a touch-sensitive device 133, which inputs instructions by touching the operation surface. The input device 13 converts instructions input by the user into electrical signals and outputs the electrical signals to the control unit 180. Note that the input device 13 is not limited to physical operation devices such as the mouse 131 and keyboard 132. The input device 13 may also include, for example, a receiving port that receives electrical signals input from an external input device.

The output device 14 is a device for presenting information to the user who owns the terminal device 10. The output device 14 is realized, for example, by a display 141. The display 141 displays data such as images, videos, and text in accordance with the control of the control unit 180. The display 141 is realized, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

The memory unit 170 is realized, for example, by memory 15 and storage 16, and stores data and programs used by the terminal device 10.

The control unit 180 is realized when the processor 19 reads a program stored in the storage unit 170 and executes the instructions contained in the program. The control unit 180 controls the operation of the terminal device 10. Specifically, for example, the control unit 180 performs the functions of an operation reception unit 181, a transmission/reception unit 182, and a display control unit 183.

The operation reception unit 181 performs processing to receive user operations input from the input device 13. The transmission/reception unit 182 performs processing to enable the terminal device 10 to transmit and receive data to and from external devices such as the server 20 in accordance with a communication protocol. The display control unit 183 controls the output device 14 to present various information to the user.

<1.2 Functional configuration of the server>
3 is a block diagram showing the functional configuration of the server 20 included in the system 1 of this embodiment. As shown in FIG. 3, the server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

The communication unit 201 performs processing that enables the server 20 to communicate with external devices.

The memory unit 202 stores data and programs used by the server 20. For example, the memory unit 202 stores a test case database (DB) 281 and a task database (DB) 282.

Test case DB281 is a database for storing information about test cases.

Task DB282 is a database for managing tasks that have occurred. In task DB282, tasks are managed, for example, on a ticket-by-ticket basis.

The control unit 203 is realized when the processor 29 reads a program stored in the memory unit 202 and executes the instructions included in the program. The control unit 203 controls the operation of the server 20. Specifically, for example, the control unit 203 performs the functions of an operation reception unit 2031, a transmission/reception unit 2032, a memory control unit 2033, a presentation unit 2034, an analysis unit 2035, a setting unit 2036, and a prediction unit 2037.

The operation reception unit 2031 receives operations from the user. For example, the operation reception unit 2031 receives the specification of test cases by testers as users, and input of test implementation results. The operation reception unit 2031 also receives, for example, input on tickets by developers as users.

The transmission/reception unit 2032 controls the process by which the server 20 transmits and receives data to and from external devices such as the terminal device 10 in accordance with a communication protocol.

The memory control unit 2033 controls the storage of information in the memory unit 202. Specifically, for example, when a tester inputs information about a test case that has already been tested, the memory control unit 2033 stores the input information in the test case DB 281. When a developer inputs a response to the results of the test that has been conducted, the memory control unit 2033 stores information about the input response in the test case DB 281. The memory control unit 2033 stores information obtained by analyzing the results of the conducted test in the test case DB 281. The memory control unit 2033 stores prediction information based on the analysis results of the conducted test in the test case DB 281.

In addition, when information about a task is input, for example, by a tester or developer, the storage control unit 2033 stores the input information in the task DB 282.

The presentation unit 2034 presents the information stored in the test case DB 281 in a format desired by the user. Specifically, the presentation unit 2034 displays a project selection screen on the display 141 of the terminal device 10. The presentation unit 2034 also displays a test suite selection screen on the display 141. In this embodiment, a test suite includes, for example, multiple test case data for a specific function. Test case data is, for example, data in which test cases defined in a test design document are converted into a format that enables test implementation management. Each of the multiple test case data has a common data structure. A test suite includes, for example, a set of test case data that is easy for a tester to implement tests. A set of test case data that is easy for a tester to implement tests is, for example, test case data for the same function or test case data for the same module.

The presentation unit 2034 also causes the display 141 to display a screen that allows the user to input the results of the test for the test case. The presentation unit 2034 also causes the display 141 to display a screen that displays analysis information based on the results of the test. The presentation unit 2034 also causes the display 141 to display a screen that displays prediction information based on the results of the test.

The analysis unit 2035 analyzes information related to the results of the tests. Specifically, the analysis unit 2035 analyzes the results of the tests and calculates the number of man-hours required, productivity, number of defects, defect rate, etc. The analysis unit 2035 also analyzes the results of the tests and extracts test cases that are likely to cause defects. Test cases that are likely to cause defects can be said to be test cases that should be tested with priority, for example. The analysis unit 2035 extracts test cases that should be tested with priority based on, for example, numerical values related to the number of defects that occur (number of defects, defect rate, etc.), productivity, etc.

The analysis unit 2035 also analyzes responses from developers to the test results and extracts test cases that are likely to cause defects, i.e., test cases that should be prioritized for testing. The analysis unit 2035 extracts test cases that should be prioritized based on, for example, the numerical value related to the amount of work required to fix any defects that occur and the cause of the defects that occur.

The setting unit 2036 sets the order in which tests should be performed on test suites or test case data. For example, before starting the testing process for a project, the setting unit 2036 sets the order in which tests should be performed on test suites or test case data. Specifically, the setting unit 2036 assigns a priority order to test suites that correspond to important functions in the software being developed, and to the test case data included in these test suites. The setting unit 2036 also assigns a priority order to test suites or test case data based on the results of identical or similar tests conducted in the past.

The setting unit 2036 also uses, for example, the results of tests already performed in the testing process to set the order in which test suites or test case data should be performed. Specifically, the setting unit 2036 assigns a priority order to data about test cases extracted by the analysis unit 2035 and to test suites that include this test case data.

The prediction unit 2037 predicts the trend of tests when they are performed in the order selected by the user. Specifically, for example, the prediction unit 2037 predicts the number of defects, including defects that occur in test cases that have not yet been performed, or the defect rate, based on test cases that have already been performed. Furthermore, the prediction unit 2037 predicts the number of defect repair man-hours, including the number of defect repair man-hours for defects that occur in test cases that have not yet been performed, and the number of defect check man-hours after repair, based on the number of defect repair man-hours for defects that have occurred in test cases that have already been performed and the number of defect check man-hours after repair.

<2 Data Structure>
FIG. 4 is a diagram showing the data structure of the test case DB 281 stored in the server 20. The test case DB 281 shown in FIG. 4 includes the items "perspective,""factor," and "level," which are used to define test cases in the test design document. "Perspective" represents the standpoint (way of thinking) on which the test is based. "Factor" represents a factor that can affect changes in the computer's state. "Level" represents a specific element of a "factor." FIG. 4 shows a case in which tests have already been conducted for factor A-levels AA to AC of perspective A and factor B-levels BA to BC of perspective B for function A. Note that FIG. 4 is an example and does not exclude data not listed.

The test case DB281 shown in FIG. 4 is constructed, for example, for the test cases of a specific project. If there are multiple projects, the test case DB281 may be constructed, for example, for each project.

As shown in Figure 4, each record in test case DB281 includes the following items: "Screen", "Function", "Person in charge", "Module", "Perspective", "Factor", "Level", "Total number of cases", "Number of cases performed", "Performance man-hours", "Productivity", "Cause of defect", "Actual results", "Prediction", etc.

The "Screen" item stores information about the screen on which the function is displayed. Information about the screen includes, for example, the name of the screen or a link to access the screen data.

The "Function" item stores information about the functions possessed by the software. Information about the function includes the name of the function, such as "login function." The function name may be entered arbitrarily by the person in charge, or may be selected from pre-registered names.

The "Person in Charge" item stores the name of the developer of the function. Note that this is not limited to a name; it can be anything that identifies the developer. It can also be an identification number, address, etc.

The "Module" item stores the module that the tester operates when conducting the test.

The "Perspective" item stores the viewpoint (concept) that the test is based on. If the "Function" item is a "login function," the "Perspective" item may store, for example, "Successful authentication," "Authentication check," "Password retention," etc.

The "Factor" item stores factors that may affect changes in the computer's status. If the "Perspective" item is "Successful Authentication," the "Factor" item may store, for example, "Whether or not there is an ID/PW mismatch."

The "Level" item stores specific operations grouped together in the "Factor." If the "Factor" item is "ID/PW inconsistency," the "Level" item includes, for example, "ID inconsistent," "PW inconsistent," "Both ID and PW inconsistent," and "ID and PW both consistent."

The "Total Cases" item stores the total number of test cases that occur at each level.

The "Number of Implemented Cases" item stores the number of test cases that have been implemented. In the test case DB 281 shown in Figure 4, all test cases have been implemented for factor A-levels AA to AC of perspective A in function A, and factor B-levels BA to BC of perspective B.

The "Man-hours Implemented" item stores the man-hours required to implement the associated test case. Man-hours may be obtained based on a report from the tester, or may be obtained automatically based on the tester's operation log.

The "Productivity" item stores the time taken for one test case. In the example shown in Figure 4, the number of minutes taken for one test case is stored.

The "Defect Cause" item stores the number of occurrences of each cause of defects that occurred during testing. The "Defect Cause" item contains multiple sub-items. The causes of defects that occurred are assigned to the sub-items. The causes assigned to the sub-items are assigned a priority according to the process in which the cause is embedded.

In software development, the work processes proceed in the following order, for example: requirements definition, basic design, detailed design, coding, and testing. There is a possibility that defects may be embedded in each of these development processes. If a defect is embedded in an upstream process, the longer it is delayed in detecting it in the testing process, the greater the cost of fixing the defect. Furthermore, the longer it is delayed in detecting it in the testing process, the greater the risk of degradation occurring. Therefore, by setting priorities for the causes of defects according to the process in which the causes are embedded, it becomes possible to understand the importance of resolving the defect that has occurred.

For example, defect factors assigned to sub-items are classified into factors originating from upstream processes, factors originating from midstream processes, and factors originating from downstream processes. Factors originating from upstream processes are, for example, factors related to defects embedded in requirements definition, and have a higher priority than factors originating from midstream processes and factors originating from downstream processes. Factors originating from midstream processes are, for example, factors related to defects embedded in basic design and detailed design, and have a higher priority than factors originating from downstream processes. Factors originating from downstream processes are, for example, factors related to defects embedded in coding and testing.

Causes of defects include, for example, "omission of requirements," "failure to consider the scope of impact," "misunderstanding between subsystems," "failure to consider specifications," "model dependency," "programming error," "misconfiguration error," "failure to conduct unit testing," and "existing bugs." Of these, for example, "omission of requirements" and "failure to consider the scope of impact" correspond to factors originating in upstream processes, "misunderstanding between subsystems," "failure to consider specifications," and "model dependency" correspond to factors originating in midstream processes, and "programming error," "misconfiguration error," "failure to conduct unit testing," and "existing bugs" correspond to factors originating in downstream processes. Note that, although the processes are classified into three categories above, the number of categories is not limited to three and can be two, four, or more.

In the example shown in Figure 4, the sub-items include "Omission of requirements," "Impact consideration," "Inter-system awareness," "Specification consideration," "Work error," and "Environmental differences." "Omission of requirements" and "Impact consideration" belong to factors originating from upstream processes, "Inter-system awareness" and "Specification consideration" belong to factors originating from midstream processes, and "Work error" and "Environmental differences" belong to factors originating from downstream processes. Note that the factors assigned to sub-items are not limited to these. Other factors may be added, or some factors may be absent. Also, some factors may be replaced with other factors.

The "Results" item stores the results of tests that have been conducted. The "Results" item includes multiple sub-items. In the example shown in Figure 4, the "Results" item includes the sub-item "Number of Defects," the sub-item "Defect Rate," the sub-item "Repair Man-hours," and the sub-item "Post-Repair Verification Man-hours."

The "Number of Defects" sub-item stores the number of defects that occurred among the corresponding test cases. The "Defect Rate" sub-item stores the probability that a defect occurred. The information displayed in the "Number of Defects" and "Defect Rate" sub-items is based on, for example, the test results entered by the tester.

The sub-item "Man-hours to fix" stores the man-hours required to fix the defect. The information represented in the sub-item "Man-hours to fix" is based, for example, on information entered by a developer after the program has been fixed. The sub-item "Man-hours to verify after fix" stores the man-hours required to test the program after the defect has been fixed. The information represented in the sub-item "Man-hours to verify after fix" is based, for example, on the results of testing the program after the fix, entered by a tester.

The "Prediction" item represents the predicted test results when tests are conducted according to the proposed priority order. The "Prediction" item includes multiple sub-items. In the example shown in Figure 4, the "Prediction" item includes the sub-item "Number of Defects," the sub-item "Defect Rate," the sub-item "Repair Man-hours," and the sub-item "Post-Repair Verification Man-hours." The information represented by the sub-item "Number of Defects," the sub-item "Defect Rate," the sub-item "Repair Man-hours," and the sub-item "Post-Repair Verification Man-hours" is calculated, for example, based on the results of tests that have already been conducted.

The test case DB 281 shown in FIG. 4 also includes an item "function importance" and an item "priority".
The item "function importance" stores the importance of the function of the software being developed. In the example shown in Figure 4, the importance is expressed as "high,""medium," or "low," but is not limited to this. The importance may also be expressed as a number.

The "Priority" item represents the priority of a test case. The "Priority" item includes multiple sub-items. In the example shown in Figure 4, the "Priority" item includes the sub-items "Plan," "Modify 1," and "Modify 2." The "Plan" sub-item stores the priority set before the start of the testing process. The "Modify 1" sub-item stores the priority set at a specific timing during the testing process based on the results of tests that have already been conducted. In the example shown in Figure 4, the "Modify 1" sub-item stores the priority set after conducting tests for test cases set as priority 1 in the "Plan" sub-item, for example, testing factor A of perspective A in function A and factor B of perspective B. The "Modify 2" sub-item stores the priority set after conducting tests for test cases set as priority 1 in the "Modify 1" sub-item, for example, testing factor A of perspective D in function C.

<3 Operation>
The operation of the system 1 when a test is administered by a test administrator will now be described.

3.1 Test Implementation
FIG. 5 is a diagram illustrating the operation of the system 1 when a test is administered by a tester.

First, the tester, who operates terminal device 10-1, operates terminal device 10-1 to access server 20. The tester operates terminal device 10-1 to display information about the project. In response to the tester's operation, terminal device 10-1 requests information about the project from server 20.

In response to a request from the terminal device 10-1, the server 20 transmits information about the project to the terminal device 10-1.

The terminal device 10-1 presents the information received from the server 20 to the tester. Specifically, for example, the terminal device 10-1 displays a project list on the display 141.

Figure 6 is a schematic diagram showing an example of a project list displayed on display 141. The project list displays a list of projects currently underway. In the example shown in Figure 6, two projects are displayed. The record representing each project includes, for example, the items "Project Key," "Project Name," "Current Process," "Start Date," and "Status."

The terminal device 10-1 accepts a project selection from the tester (step S101). Specifically, the tester selects, for example, a project related to software that requires testing from the projects displayed in the list. Once the project is selected by the tester, the terminal device 10-1 requests information about the test suite for the selected project from the server 20.

In response to a request from terminal device 10-1, server 20 obtains information about the test suite from test case DB 281. Server 20 transmits the obtained information to terminal device 10-1.

The terminal device 10-1 presents the information received from the server 20 to the tester. Specifically, for example, the terminal device 10-1 displays a list of test suites on the display 141.

Figure 7 is a schematic diagram showing an example of a test suite list displayed on display 141. The test suite list displays a list of test suites included in the selected project. In the example shown in Figure 7, five test suites are displayed. A record representing each test suite includes, for example, the items "No.", "Test Suite," "Responsible Person," "Priority," "Scope," "Feature," "Number of Cases," "Progress," and "Update Date." The "Priority" item has sub-items "Plan" and "Revision 1." Before testing is performed, the "Plan" sub-item stores the priority set in the planning stage. Meanwhile, the "Revision 1" sub-item is still blank.

The terminal device 10-1 accepts the test suite selection from the tester (step S102). Specifically, the tester refers to the sub-item "Plan" and selects the test suite with priority 1, that is, the test suite that should be tested with the highest priority. Once the test suite is selected by the tester, the terminal device 10-1 requests information about the test case data of the selected test suite from the server 20.

In response to a request from terminal device 10-1, server 20 transmits information about the test case data to terminal device 10-1.

The terminal device 10-1 presents the information received from the server 20 to the tester. Specifically, for example, the terminal device 10-1 displays a list of test case data on the display 141.

Figure 8 is a schematic diagram showing an example of a list of test case data displayed on display 141. The list of test case data displays the test case data included in the selected test suite in list form. In the example shown in Figure 8, five test case data are displayed. The record representing each test case data includes, for example, the items "ID", "major item", "medium item", "minor item", "implementation result", "priority", and "remarks".

The "Execution Results" item has field objects 1411 and 1412. The field object 1411 for a test case that has not been tested displays "UnExe," indicating that the test has not yet been performed.

The "Priority" item has the sub-item "Plan" and the sub-item "Fix 1." Before testing is performed, the sub-item "Plan" stores the priority set during the planning stage. In the example shown in Figure 8, test cases in a test suite with priority 1 are displayed, so priority 1 is set for all test cases. However, the sub-item "Fix 1" is still blank.

The tester refers to the "Plan" sub-item and selects one of the test cases with priority 1, i.e., the test case that should be tested with the highest priority. Once the tester selects a test case, terminal device 10-1 tests the selected test case.

The terminal device 10-1 accepts input of the test results from the tester (step S103). Specifically, for example, if the tester tests the test case with ID: 1 and no defects occur, the tester presses the field object 1411, which changes the display of the field object 1411 to indicate that no defects occurred. As a result, the display of the field object 1411 becomes, for example, "OK."

The terminal device 10-1 transmits the test results indicating that no defects occurred and the man-hours required for the test to the server 20. The server 20 stores the test results and the man-hours required in the test case DB 281 using the storage control unit 2033. The server 20 also analyzes the test results using the analysis unit 2035, and calculates, for example, the man-hours required, productivity, number of defects, and defect rate.

When the tester selects "OK" for the display of field object 1411, they can begin testing the next test case, for example, test case ID: 2.

On the other hand, if a defect occurs when testing a test case with ID: 1, the tester presses field object 1411, which changes the display of field object 1411 to indicate that a defect has occurred. As a result, the display of field object 1411 becomes, for example, "NG."

The terminal device 10-1 transmits the test results indicating that a defect has occurred and the man-hours required for the test to the server 20. The server 20 stores the test results and the man-hours required in the test case DB 281 using the storage control unit 2033. The server 20 also analyzes the test results using the analysis unit 2035, and calculates, for example, the man-hours required, productivity, number of defects, and defect rate.

When the tester switches the display of field object 1411 to "NG," he or she creates a ticket to be issued to the developer. A ticket is a unit for managing work in a project. In this embodiment, tickets are used, for example, to share information about the occurrence of a defect between the tester and the developer, and to manage program modification as a task for the developer. Specifically, for example, the tester presses field object 1411 to switch the display to "NG," and then presses field object 1412.

When the field object 1412 is pressed, the terminal device 10-1 displays the ticket input form on the display 141.

Figure 9 is a schematic diagram showing an example of a ticket input form displayed on display 141. In the example shown in Figure 9, an input window 1413 is displayed on display 141. Input window 1413 has boxes for entering, for example, "Process," "Type," "Title," "Priority," "Severity," "Assignee," "Reporter," and "Due date."

The ongoing process is entered in "Process." Entry is made in a pull-down format, where options are displayed by clicking the downward-pointing triangle on the right edge of the box. In the example shown in Figure 9, the ticket is being issued in the test process, so "Test" is entered.

The type of ticket is entered in "Type." Entry is made, for example, using a pull-down menu. In the example shown in Figure 9, the ticket is being issued to have the developer fix the bug, so "Bug" is entered.

The title of the ticket is entered in "Title." For example, any content can be entered from the tester. In the example shown in Figure 9, the title "I tried BBB with AAA, but a problem occurred where CCC was not performed" is entered.

"Priority" is the input for the priority of the task created by issuing a ticket. "Severity" is the input for the importance of the task created by issuing a ticket. "Assignee" is the input for the person in charge of the task created by issuing a ticket. This is generally the software developer. "Reporter" is the input for the creator of the ticket. These inputs are made, for example, in a pull-down format.

The "Due date" field is where the deadline for the task created by issuing a ticket is entered. This can be entered directly by the tester, or the date can be specified via a calendar.

When the tester has finished entering the content to be included in the ticket, he or she presses field object 1414. When field object 1414 is pressed, terminal device 10-1 transmits the content entered by the tester to server 20 and requests server 20 to register the transmitted content as a ticket. After registering the ticket on server 20, the tester begins testing the next test case, for example, test case ID: 2.

When the server 20 receives a request from the terminal device 10-1, the storage control unit 2033 stores the received content in the task DB 282 and registers it as a new ticket. The server 20 also issues the registered ticket to the developer listed in "Assignee" (step S201).

When a ticket is issued by the server 20, the terminal device 10-2 operated by the developer notifies the developer that the ticket has been issued. Upon receiving the notification, the developer displays the contents of the issued ticket on the display 141 of the terminal device 10-2.

Figure 10 is a schematic diagram showing an example of the contents of a ticket displayed on the display 141. After displaying the contents of the ticket to the developer, the terminal device 10-2 accepts input from the developer (step S104). Specifically, for example, if the developer checks the displayed contents and determines that a defect has occurred in a test for a specific test case, they investigate the cause of the defect. After identifying the cause, the developer enters the defect cause in the area of the ticket where the defect cause is entered. In the example shown in Figure 10, "Relative Issue" corresponds to the area where the defect cause is entered. Once the defect cause has been entered in "Relative Issue," the terminal device 10-2 considers the ticket contents to have been updated and transmits the entered information to the server 20. After the developer enters the defect cause in the ticket, they begin repairing the software where the defect occurred.

When the server 20 receives the information from the terminal device 10-2, the storage control unit 2033 updates the task DB 282 based on the received information. The server 20 also notifies the tester listed in "Reporter" that the ticket has been updated (step S202).

The tester repeats steps S102 to S202 until there are no test cases.

<3.2 Implementation of modification confirmation tests>
In step S104 shown in FIG. 5, the developer inputs the cause of the defect into the ticket, and then starts repairing the software in which the defect occurred.

Figure 11 is a diagram that explains the operation of system 1 when software is modified by a developer and the modified software is tested by a tester.

The terminal device 10-2 receives input from the developer reporting that the software repair has been completed (step S105). Specifically, for example, the developer enters the details of the repair of the defect and the number of man-hours required to repair the defect into the ticket. When the details of the repair of the defect and the number of man-hours required to repair the defect are entered, the terminal device 10-2 considers the ticket contents to have been updated and transmits the entered information to the server 20.

When the server 20 receives the information from the terminal device 10-2, the storage control unit 2033 updates the task DB 282 based on the received information. The server 20 also notifies the tester listed in "Reporter" that the ticket has been updated (step S203).

When the terminal device 10-1 receives notification from the server 20 that the ticket has been updated, it notifies the tester that the ticket has been updated. Upon receiving the notification, the tester displays the contents of the updated ticket on the display 141 of the terminal device 10-1. The tester refers to the displayed contents and re-tests the test case in which the defect occurred.

The terminal device 10-1 accepts input of the test results from the tester (step S106). Specifically, for example, if the tester tests a test case for which the field object 1411 is "NG" and no problems occur, the tester presses the field object 1411 to change the display of the field object 1411 to "OK."

The terminal device 10-1 transmits the implementation results indicating that the defect has been resolved and the man-hours required for post-repair verification to the server 20. The server 20 then stores the test implementation results and the man-hours required for post-repair verification in the test case DB 281 via the storage control unit 2033.

For example, if a problem occurs during testing of a test case where field object 1411 is "NG," the tester presses field object 1412 to create a ticket. Once the tester has finished entering the content to be included in the ticket, he or she presses field object 1414. When field object 1414 is pressed, terminal device 10-1 transmits the content entered by the tester to server 20 and requests server 20 to register the transmitted content as a ticket.

When the server 20 receives a request from the terminal device 10-1, the storage control unit 2033 stores the received content in the task DB 282 and registers it as a new ticket. The server 20 also issues the registered ticket to the developer listed in "Assignee."

3.3 Priority setting
FIG. 12 is a flowchart showing an example of the operation of the server 20 when setting priorities for test suites.

First, the analysis unit 2035 determines whether the tester has instructed the tester to reset the priorities (step S21). For example, a field object for instructing the tester to reset the priorities is displayed (not shown) in the test suite list display shown in FIG. 7 or the test case data list display shown in FIG. 8. If there is a specific reason, the tester requests the server 20 to reset the priorities by pressing the field object. Examples of specific reasons include a low probability of a defect occurring in the remaining test case data in the test suite currently being tested, or a desire to reset the priorities.

If there is an instruction to reset the priority (Yes in step S21), the analysis unit 2035 proceeds to step S23. If there is no instruction to reset the priority (No in step S21), the analysis unit 2035 determines whether predetermined requirements have been met for the tests of the test case data (step S22). The predetermined requirements include, for example, whether enough tests have been performed to analyze the results of the tests, and specifically, the following:
- Testing of all test case data included in the selected test suite has been performed, causes of any defects that have occurred have been entered, or the defects that have occurred have been resolved. - Testing of all test case data related to a specified perspective in the selected test suite has been performed, causes of any defects that have occurred have been entered, or the defects that have occurred have been resolved. - Testing of all test case data related to a specified factor in the selected test suite has been performed, causes of any defects that have occurred have been entered, or the defects that have occurred have been resolved. - Testing of all test case data related to a specified level in the selected test suite has been performed, causes of any defects that have occurred have been entered, or the defects that have occurred have been resolved. - Testing of a predetermined number of test case data in the selected test suite has been performed, causes of any defects that have occurred have been entered, or the defects that have occurred have been resolved. - Testing of a predetermined number of test case data has been performed, causes of any defects that have occurred have been entered, or the defects that have occurred have been resolved. - A predetermined amount of time has passed within the predetermined test period.

If the specified requirements for the test case data test are met (Yes in step S22), the analysis unit 2035 proceeds to step S23. If the specified requirements for the test case data test are not met (No in step S22), the analysis unit 2035 repeats the processes in steps S21 to S22.

In step S23, the analysis unit 2035 analyzes the results of the tests that have been performed and extracts, from the remaining test cases, test cases that should be prioritized for testing. In the example shown in FIG. 4, when tests are performed on test cases included in a test suite that was set to priority 1 during planning, the analysis unit 2035 analyzes the results of these tests and extracts, from the remaining test cases, test cases that should be prioritized for testing. Note that in FIG. 4, the performance of tests on test cases included in a test suite that was set to priority 1 during planning can be rephrased as the performance of tests on a predetermined number of test case data.

FIG. 13 is a flowchart showing a specific example of the operation of the analysis unit 2035.
First, the analysis unit 2035 reads out the person in charge, module, viewpoint, factor, level, defect cause, and performance for the test case data for which the test has been performed from the test case DB 281 (step S231).

Based on the implementation results, the analysis unit 2035 selects test cases that should be prioritized for testing from among the test cases that have already been tested (step S232). Specifically, for example, the analysis unit 2035 selects test cases that should be prioritized for testing by taking into consideration a numerical value related to the number of defects that occur (number of defects, defect rate, etc.), a numerical value related to the labor hours required to repair the defects that have occurred, the project process from which the cause of the defects that have occurred originates, or a combination of at least two of these.

More specifically, for example, if there is a test case group in which the numerical value relating to the number of defects that occur or the numerical value relating to the labor required to fix the defects that occur is higher than the limit value requiring special measures, the analysis unit 2035 selects the test case group for which that numerical value has been calculated. In this embodiment, a test case group refers to a group of test cases that are treated as test cases of similar nature, and is expressed, for example, by factor or level. If the numerical value relating to the number of defects that occur and the numerical value relating to the labor required to fix the defects that occur do not exceed the limit value, the analysis unit 2035 selects, for example, a test case group in which a defect occurred due to a factor originating in an upstream process of the project.

In the example shown in FIG. 4, the analysis unit 2035 selects a test case group related to factor A, which has been confirmed to be caused by factors such as "missing requirements" and "impact consideration," originating from upstream processes, as there are no test case groups for which the numerical values related to the number of defects that occur and the numerical values related to the labor hours required to fix the defects that have occurred exceed the limit values. Furthermore, the analysis unit 2035 selects a test case group related to factor B, which has been confirmed to be caused by "work error," originating from downstream processes, as the test case group that should be given the next highest priority for testing.

Note that the selection of test cases may also be performed based on other elements that define the test cases. For example, the analysis unit 2035 may select test cases based on a perspective or level.

The analysis unit 2035 identifies the characteristics of the selected test cases (step S233). For example, the analysis unit 2035 identifies the elements that define each selected test case, such as screens, functions, personnel in charge, modules, perspectives, factors, levels, etc.

In the example shown in FIG. 4, the analysis unit 2035 identifies "Screen A, Function A, Person in Charge AAAA, Module A, Viewpoint A, Factor A, Level AA to Level AC" as the characteristics of a group of test cases in which a defect occurred due to a factor originating in an upstream process. Furthermore, the analysis unit 2035 identifies "Screen A, Function A, Person in Charge AAAA, Module A, Viewpoint B, Factor B, Level BA to Level BC" as the characteristics of a group of test cases in which a defect occurred due to a factor originating in a downstream process.

The analysis unit 2035 extracts test cases that match the identified characteristics or similar test cases from the untested test cases (step S234). A matching test case refers to a test case in which all elements that define the test case, such as "screen, function, person in charge, module, viewpoint, factor, and level," match the identified characteristics of the executed test case. A similar test case refers to a test case in which at least one of the elements that define the test case, such as "screen, function, person in charge, module, viewpoint, factor, and level," matches the identified characteristics of the executed test case.
The analysis unit 2035 may vary the degree of similarity depending on the matching elements. For example, if the "factors, levels" match, the analysis unit 2035 determines that the contents of the test cases are close and evaluates the degree of similarity as high. On the other hand, if the viewpoints, modules, or personnel match, the analysis unit 2035 evaluates that although there is a relationship, the degree of similarity is lower than when the "factors, levels" match.

In the example shown in FIG. 4, the analysis unit 2035 extracts test cases with matching "factors and levels" from untested test cases as test case groups that are highly similar to the identified features of "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Levels AA to AC." In other words, the analysis unit 2035 extracts the test case group of "Screen B, Function C, AAAA, Module A, Viewpoint D, Factor A, Levels AA to AC" from among the untested test cases as a similar test case group.

In addition, the analysis unit 2035 extracts the test case group "Screen B, Function B, BBBB, Module C, Viewpoint A, Factor B, Levels BA to BC" and the test case group "Screen A, Function A, AAAA, Module B, Viewpoint C, Factor C, Levels CA to CC" as test case groups with low similarity to the identified features of "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Levels AA to AC."

The analysis unit 2035 also extracts test cases with matching "factors and levels" from untested test cases as test cases that are highly similar to the identified features of "Screen A, Function A, AAAA, Module B, Viewpoint B, Factor B, Level BA to Level BC." In other words, the analysis unit 2035 extracts test cases with "Screen B, Function B, BBBB, Module C, Viewpoint A, Factor B, Level BA to Level BC" from among untested test cases as similar test cases.

In addition, the analysis unit 2035 extracts the test case group "Screen A, Function A, AAAA, Module B, Viewpoint B, Factor B, Level BA-Level BC" as a group of test cases with low similarity to the identified features of "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Level AA-Level AC" and the test case group "Screen A, Function A, AAAA, Module B, Viewpoint C, Factor C, Level CA-Level CC."

The analysis unit 2035 may make a similarity determination based on a functional specification associated with the test case or test suite. A functional specification is data that describes the specifications for the functions possessed by the test target. A test design document that defines a test case is created with reference to the functional specification. Information about the functional specification referenced during creation is associated with the test case or test suite. If a functional specification is associated with the test case selected in step S232, the analysis unit 2035 may extract test cases associated with the same functional specification from untested test cases as similar test cases.

Returning to the explanation of Figure 12, the setting unit 2036 references the test cases extracted in step S23 and sets priorities for test cases that have not yet been tested (step S24).

Specifically, the setting unit 2036 sets a high priority for test cases that match or are highly similar to test cases that should be given priority for testing. More specifically, the setting unit 2036 sets the priority by comprehensively taking into consideration the reasons why a test should be given priority, the degree of similarity to the selected test case, etc. Reasons why a test should be given priority include various factors, such as circumstances specific to the project, the amount of repair work required, the defect rate, and variations in the skills of the people in charge.

In the example shown in FIG. 4, the setting unit 2036 sets a priority of 1 for the extracted test case group of "Screen A, Function A, AAAA, Module A, Viewpoint A, Factor A, Levels AA to AC." The setting unit 2036 may also set a priority for each level in this test suite based on the defect causes, defect rate, repair labor hours, etc. of tests that have already been performed. For example, the setting unit 2036 may set a priority of 1 for the test case group of level AA, a priority of 2 for the test case group of level AC, and a priority of 3 for the test case group of level AB.

The setting unit 2036 also sets a priority of 2 for the extracted test case group of "Screen B, Function B, BBBB, Module C, Viewpoint A, Factor B, Level BA to Level BC." The setting unit 2036 may also set a priority for each level in this test suite based on the defect causes, defect rate, repair labor hours, etc. of tests that have already been performed. For example, the setting unit 2036 may set a priority of 1 for the test case group of level BA, a priority of 2 for the test case group of level BC, and a priority of 3 for the test case group of level BB.

Furthermore, the setting unit 2036 assigns a priority of 3 to "Screen A, Function A, AAAA, Module B, Viewpoint C, Factor C, Levels CA to CC" as they have a low degree of similarity.

In addition, depending on the track record of the selected test case, the cause of the defect, and the degree of similarity of the test cases extracted from the untested test cases, the test cases may be assigned the same priority.

The storage control unit 2033 stores the set priority in the test case DB 281 as the corrected priority.
14 to 17 are schematic diagrams showing examples of what is displayed on the display 141 when priorities are corrected based on the results of tests that have already been carried out.

Figure 14 is a schematic diagram showing an example of a list of test suites displayed on display 141. The example shown in Figure 14 represents the display screen of display 141 when the priorities are modified by setting unit 2036 after testing of the test suite for function A-1 is completed. According to the example shown in Figure 14, the priority of the test suite for function C, which was priority 4, becomes 1; the priority of the test suite for function B, which was priority 3, becomes 2; the priority of the test suite for function A-2, which was priority 2, becomes 3; and the priority of the test suite for function D, which was priority 5, becomes 4.

Figure 15 is a schematic diagram showing an example of a list of test case data displayed on the display 141. The example shown in Figure 15 shows a list of test case data when a test suite for function C with priority 1 is selected in Figure 14. According to the example shown in Figure 15, the priority of the test case data that was priority 4 has been changed to priority 1.

Figure 16 is a schematic diagram showing an example of a list of test suites displayed on the display 141. The example shown in Figure 16 represents the display screen of the display 141 when the priorities are modified by the setting unit 2036 after testing of the test suite for function C is completed. According to the example shown in Figure 16, the priority of the test suite for function B, which was priority 2, is now 1; the priority of the test suite for function A-2, which was priority 3, is now 2; and the priority of the test suite for function D, which was priority 4, is now 3.

Figure 17 is a schematic diagram showing an example of a list of test case data displayed on the display 141. The example shown in Figure 17 shows a list of test case data when the test suite for function B with priority 1 is selected in Figure 16. According to the example shown in Figure 17, the priority of the test case data that was priority 2 has been changed to priority 1.

3.4 Test Trend Prediction
FIG. 18 is a flowchart showing an example of the operation of the server 20 when predicting test trends.

When a tester wants to check the test trends, i.e., the trend in the number of remaining test cases, predicted information on the trend in the number of defects, etc., the tester presses a field object to instruct the display of test trends. The field object is displayed, for example, on display 141.

When the field object is pressed, the prediction unit 2037 determines whether the number of test cases that have already been tested satisfies a predetermined requirement (step S25). The predetermined requirement includes, for example, whether a sufficient number of tests have been conducted to enable the trend of testing to be predicted, and specifically, the following:
- Testing has been performed on all test case data included in any test suite, or any defects that have occurred have been resolved. - Testing has been performed on all test case data related to a specified perspective in any test suite, or any defects that have occurred have been resolved. - Testing has been performed on all test case data related to a specified factor in any test suite, or any defects that have occurred have been resolved. - Testing has been performed on all test case data related to a specified level in any test suite, or any defects that have occurred have been resolved. - Testing has been performed on a predetermined number of test case data in any test suite, or any defects that have occurred have been resolved. - Testing has been performed on a predetermined number of test case data in any test suite, or any defects that have occurred have been resolved.

If the number of test cases that have already been tested meets the specified requirements (Yes in step S25), the analysis unit 2035 identifies the characteristics of the test cases that have been tested (step S26). For example, the analysis unit 2035 identifies the elements that define the test cases that have been tested, such as the screen, function, person in charge, module, perspective, factor, and level. If the number of test cases that have already been tested does not meet the specified requirements (No in step S25), the analysis unit 2035 repeats the processing of step S25 until the number of test cases meets the requirements.

The analysis unit 2035 extracts test cases that match or are similar to the identified characteristics from the untested test cases (step S27). For example, the analysis unit 2035 extracts test cases from the untested test cases whose screens, functions, responsible parties, modules, perspectives, factors, and levels match the screens, functions, responsible parties, modules, perspectives, factors, and levels identified for the executed test cases. The analysis unit 2035 also extracts test cases from the untested test cases whose at least one of the screens, functions, responsible parties, modules, perspectives, factors, and levels matches the screens, functions, responsible parties, modules, perspectives, factors, and levels identified for the executed test cases.

The prediction unit 2037 calculates various parameters representing test trends for the extracted test cases based on the performance of the executed test cases and the causes of defects (step S28). Specifically, the prediction unit 2037 calculates the number of defects, defect rate, labor hours, repair labor hours, or post-repair verification labor hours for the extracted test case group based on the performance of the executed test cases.

For example, the prediction unit 2037 sets a defect rate for a test case group corresponding to a test case group with a specified factor and level among the executed test cases, and a test case group with the same factor and level. The prediction unit 2037 may also correct the defect rate based on factors other than the factor and level. The prediction unit 2037 multiplies the number of cases in the test case group with the same factor and level by the set defect rate to calculate the number of defects for this test case group. The prediction unit 2037 calculates the man-hours, repair man-hours, or post-repair confirmation man-hours for this test case group, for example, based on the ratio between the number of defects in the corresponding test case group and the number of defects in the test case group with the same factor and level.

If there is a test case group that has the same factors and levels as a test case group in which a defect occurred due to a factor originating in an upstream process among the untested test cases, the prediction unit 2037 may calculate the number of defects predicted to occur from this test case group so that the later the test is performed, the greater the number of defects.

The prediction unit 2037 calculates the trend in the number of defects, the trend in the remaining test cases, the trend in the number of recovered defects, or the trend in the number of recovered cases over the period from the start of the testing process based on the number of defects, man-hours, repair man-hours, or post-repair verification man-hours for the tests that have been performed, and the number of defects, man-hours, repair man-hours, or post-repair verification man-hours predicted for the test cases that have not yet been performed. At this time, the prediction unit 2037 calculates the trend when tests are performed in the order according to the set priorities.

The storage control unit 2033 stores the calculated parameters as predicted parameters in the test case DB 281.

The presentation unit 2034 presents the test progress to the user based on the calculated parameters (step S29). Specifically, for example, the presentation unit 2034 displays a graph showing the test progress on the display 141.

Figure 19 is a schematic diagram showing an example of a graph indicating test trends, displayed on display 141. In the diagram shown in Figure 19, the horizontal axis represents the test period, and the vertical axis represents the remaining number or number of occurrences. In Figure 19, the solid line represents the number of defects, the dashed line represents the number of defect fixes, the one-dot chain line represents the remaining number of cases, and the two-dot chain line represents the remaining number of fix confirmations.

As described above, in this embodiment, the server 20 uses the storage control unit 2033 to store the results of tests performed in the testing process in the test case DB 281. The analysis unit 2035 selects test cases that should be prioritized for testing from among the test cases that have been tested based on the results. The analysis unit 2035 extracts test cases similar to the selected test case from among the untested test cases in the testing process. The setting unit 2036 then sets the priority for testing the extracted test cases higher than that of other untested test cases. This enables the server 20 to set tests that should be prioritized from among the untested tests based on the results of tests performed in the testing process. Furthermore, because it is possible to start with tests that are more likely to cause defects, the chain reaction of new bugs occurring due to software modifications can be prevented. In other words, the risk of degradation is reduced.

In addition, in this embodiment, the analysis unit 2035 selects a test case that should be given the next highest priority from among the test cases that have already been tested. The analysis unit 2035 extracts test cases that are similar to the selected test case that should be given the next highest priority from among the test cases that have not yet been tested. The setting unit 2036 then sets the priority for testing the extracted test cases to be lower than the priority that was previously set. This allows the server 20 to re-set the order in which untested tests will be performed based on the results of the tests performed in the testing process.

In addition, in this embodiment, the analysis unit 2035 extracts test cases that match any of the elements that define a tested test case as similar test cases from untested test cases. Specifically, for example, the analysis unit 2035 classifies test cases with factors and levels that match the factors and levels of a tested test case as similar test cases. Also, for example, the analysis unit 2035 classifies test cases that match the functional specifications referenced when creating a tested test case as similar test cases. This enables the server 20 to extract test cases that are similar to tested test cases from untested test cases with a high degree of accuracy.

In addition, in this embodiment, the analysis unit 2035 selects test cases that should be tested with priority in response to instructions from the user. This allows the server 20 to set priorities at the timing desired by the user.

In addition, in this embodiment, once a predetermined number of tests have been performed, the analysis unit 2035 selects the test cases that should be prioritized for testing. This allows the server 20 to set priorities without any instructions from the user.

In addition, in this embodiment, the analysis unit 2035 selects the test cases that should be prioritized for testing based on the numerical value related to the number of defects that occur, which is included in the implementation results. This makes it possible to assign personnel who are skilled at reporting problems to cases with a high defect rate. Furthermore, since defects are expected to occur in the first half of the test, developers can be assigned efficiently.

In addition, in this embodiment, the analysis unit 2035 selects test cases that should be prioritized for testing based on the numerical value related to the man-hours required to fix any defects that have occurred, which is included in the implementation results. This makes it possible to test test cases that are expected to require a large number of man-hours to fix early in the testing process, thereby enabling developers to be assigned efficiently. It also makes it possible to assign dedicated personnel to cases that take a long time.

In addition, in this embodiment, the storage control unit 2033 stores the causes of defects that occurred during testing in the testing process in the test case DB 281. The analysis unit 2035 selects test cases that should be prioritized for testing based on the causes of the defects that occurred, which are included in the implementation results, specifically, the project process from which the causes originate. This makes it possible to prioritize tests that detect defects embedded in upstream processes in software development, such as the requirements definition process, design process, coding process, and testing process. This reduces the cost of repairing defects and the risk of degradation. Furthermore, the server 20 sets priorities using the results of tests conducted in the testing process within the project. This allows testers to understand the testing order appropriate for the project in real time.

In addition, in this embodiment, the presentation unit 2034 presents the set priority to the user. This allows the tester to conduct the test while being aware of the test cases with high priority.

In addition, in this embodiment, the presentation unit 2034 assigns a priority to each set of test cases (test suite) or each individual test case and presents the priority to the user. This allows the tester to efficiently access test cases that must be tested as a priority.

In addition, in this embodiment, the prediction unit 2037 predicts the test results of the extracted test cases from the test execution results. The prediction unit 2037 then predicts the test trends when untested test cases are tested in the order according to the set priorities. This allows the tester to visually grasp how the tests will progress in the testing process if the set priorities are followed. This allows the tester to intuitively determine whether the set priorities are correct.

Therefore, the server 20 according to this embodiment enables tests to be carried out efficiently and effectively during the testing process.

<Modification>
In the above embodiment, the priority of the functions to be tested is set at the time of planning. However, the priority of the functions to be tested does not have to be set at the time of planning.

Furthermore, in the above embodiment, an example was described in which test suites were displayed in the order of priority at the time of planning. However, the order in which test suites are displayed may also follow the order of priority after resetting.

In addition, in the above embodiment, an example was described in which the display control unit 183 displays the priority on the display 141. However, the display control unit 183 does not necessarily need to display the priority. The display control unit 183 may arrange the test suites or test cases in the order of the set priority and display the test suites or test cases. This reduces the burden on the tester, as performing the tests from top to bottom will result in the optimal order.

The above describes several embodiments of the present disclosure, but these embodiments can be embodied in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are considered to be within the scope of the invention and its equivalents as set forth in the claims, as well as within the scope and spirit of the invention.

<Additional Notes>
The matters described in the above embodiments will be supplemented below.

(Appendix 1)
A program to be executed by a computer having a processor 29 and a memory 25, the program causing the processor to execute the steps of storing information about tests performed in a testing process (steps S201, S202), selecting a test case to be tested as a priority from among test cases that have already been tested based on the information about the test (step S232), extracting test cases similar to the selected test case from untested test cases in the testing process (step S234), and setting the priority of testing the extracted test case higher than that of other untested test cases (step S24).

(Appendix 2)
A program described in Appendix 1, in which in the selecting step, a test case that should be tested next is selected from among test cases that have already been tested, in the extracting step, a test case similar to the selected test case that should be tested next is extracted from test cases that have not yet been tested, and in the setting step, the priority for testing the extracted test case is set lower than the set priority.

(Appendix 3)
A program described in (Appendix 1) or (Appendix 2), in which, in the extraction step, test cases that match any of the elements that define test cases that have been tested are extracted as similar test cases from test cases that have not been tested.

(Appendix 4)
A program described in Appendix 3, in which in the extraction step, test cases with factors and levels that match the factors and levels of test cases that have already been tested are considered to be similar test cases.

(Appendix 5)
In the extraction step, a test case that has already been tested and a test case whose functional specification document referenced when the test case was created match are considered to be similar test cases (Appendix 3).

(Appendix 6)
A program according to any one of (Supplementary Note 1) to (Supplementary Note 5), wherein in the selecting step, a test case to be tested with priority is selected in response to an instruction from a user (Step S21).

(Appendix 7)
A program according to any one of (Supplementary Note 1) to (Supplementary Note 5), wherein in the selecting step, when a predetermined number of tests have been performed, a test case to be tested with priority is selected (Step S22).

(Appendix 8)
A program described in any one of (Appendix 1) to (Appendix 7), wherein in the storing step, the test implementation results are stored, and in the selecting step, test cases that should be tested with priority are selected based on a numerical value related to the number of defects that occur, which is included in the implementation results.

(Appendix 9)
A program described in any one of (Appendix 1) to (Appendix 7), in which, in the storing step, the labor hours required to repair defects that occur during testing in the testing process are stored, and in the selecting step, test cases that should be tested with priority are selected based on the numerical value related to the labor hours required to repair the defects that occur.

(Appendix 10)
A program described in any one of (Appendix 1) to (Appendix 7), in which, in the storing step, the cause of a defect that occurred during testing in the test process is stored, and in the selecting step, the test case that should be tested with priority is selected based on the cause of the defect that occurred.

(Appendix 11)
In the selection step, a test case that should be tested as a priority is selected based on the project process from which the cause originates (Supplementary Note 10).

(Appendix 12)
12. The program according to claim 1, wherein the program causes a processor to execute a step (presentation unit 2034) of presenting the set priority to a user.

(Appendix 13)
The program according to claim 12, wherein in the presenting step, a priority is assigned to each set of test cases or each individual test case, and the priorities are presented to the user.

(Appendix 14)
A program described in any of (Appendix 1) to (Appendix 13) that causes a processor to execute a step (step S28) of predicting the test results of the extracted test cases from the test execution results, and a step (step S28) of predicting the test trends when tests of untested test cases are executed in an order according to the set priority.

(Appendix 15)
A method executed by a computer having a processor 29 and a memory 25, the method comprising the steps of: storing information about tests performed in a testing process; selecting a test case to be tested as a priority from among test cases that have already been tested based on the information about the test; extracting test cases similar to the selected test case from untested test cases in the testing process; and setting the priority of testing the extracted test case higher than that of other untested test cases.

(Appendix 16)
An information processing device (20) comprising a processor (29) and a memory (25), wherein the processor executes the steps of: storing information regarding tests performed in a testing process (storage control unit (2033)); selecting a test case that should be tested as a priority from among test cases that have already been tested based on the information regarding the test; extracting test cases similar to the selected test case from untested test cases in the testing process (analysis unit (2035)); and setting the priority of testing the extracted test case to be higher than that of other untested test cases (setting unit (2036)).

(Appendix 17)
A system comprising: a means (storage control unit 2033) for storing information regarding tests performed in a testing process; a step of selecting a test case that should be tested as a priority from among test cases that have already been tested based on the information regarding the test; a means (analysis unit 2035) for extracting test cases similar to the selected test case from among test cases that have not yet been tested in the testing process; and a means (setting unit 2036) for setting the priority of testing the extracted test case higher than that of other test cases that have not yet been tested.

1...System 10...Terminal device 121...Communication unit 13...Input device 131...Mouse 132...Keyboard 133...Touch-sensitive device 14...Output device 141...Display 15...Memory 16...Storage 170...Memory unit 180...Control unit 181...Operation acceptance unit 182...Transmission/reception unit 183...Display control unit 19...Processor 20...Server 201...Communication unit 202...Memory unit 203...Control unit 2031...Operation acceptance unit 2032...Transmission/reception unit 2033...Memory control unit 2034...Presentation unit 2035...Analysis unit 2036...Setting unit 2037...Prediction unit 25...Memory 26...Storage 29...Processor 80...Network

Claims (8)

A program to be executed by a computer having a processor and a memory, the program causing the processor to:
storing information about tests performed in the testing process;
selecting a test case to be tested with priority from among the test cases that have already been tested based on the information about the test;
In the testing step, a step of extracting test cases similar to the selected test case from untested test cases;
and setting a priority of testing the extracted test case higher than that of other untested test cases ;
In the selecting step, a program selects the test cases to be tested with priority based on a failure rate calculated from the test results included in the test information . In the selecting step, a group of test cases having a failure rate higher than a predetermined value is selected;
In the extracting step, a test case group similar to the selected test case group is extracted from untested test cases;
2. The program according to claim 1, wherein in the setting step, the priority of testing the extracted test cases is set higher than that of other untested test cases. In the selecting step, a test case to be tested next is selected from the test cases that have already been tested;
In the extracting step, a test case similar to the selected test case to be tested with next priority is extracted from the untested test cases;
2. The program according to claim 1, wherein in the setting step, a priority for executing a test on the extracted test case is set lower than the set priority. The program described in claim 1 or 2, wherein in the extraction step, test cases that match any of the elements that define the executed test cases are extracted as the similar test cases from the untested test cases. predicting test results of the extracted test cases from execution results of the tests;
5. The program according to claim 1, further comprising: a step of predicting a test trend when the tests of the untested test cases are performed in the order according to the set priorities. 1. A computer-implemented method comprising a processor and a memory, the method comprising:
storing information about tests performed in the testing process;
selecting a test case to be tested with priority from among the test cases that have already been tested based on the information about the test;
In the testing step, a step of extracting test cases similar to the selected test case from untested test cases;
and setting a priority of testing the extracted test case higher than that of other untested test cases ;
In the selecting step, the test cases to be tested are selected based on a failure rate calculated from the test results included in the test information . An information processing device including a processor and a memory,
the processor:
storing information about tests performed in the testing process;
selecting a test case to be tested with priority from among the test cases that have already been tested based on the information about the test;
In the testing step, a step of extracting test cases similar to the selected test case from untested test cases;
and setting a priority of testing the extracted test case higher than that of other untested test cases ;
In the selecting step, the information processing device selects the test cases to be preferentially tested based on a failure rate calculated from the test implementation results included in the information related to the test . means for storing information about tests performed in the testing process;
a means for selecting a test case to be tested with priority from among the test cases that have already been tested based on the information about the test;
a means for extracting test cases similar to the selected test case from untested test cases in the testing step;
means for setting a priority for testing the extracted test case higher than that for other untested test cases ;
The selecting means is a system that selects the test cases that should be tested with priority based on a failure rate calculated from the test execution results included in the test information .