JPH0628040B2 - Software quality automatic evaluation method - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a software (program) quality evaluation method, and more particularly to a method for automatically evaluating the quality of software by the transition of the number of bugs extracted by a test. .

[Background of the Invention]

Even if design software is fully used, bugs randomly appear in software due to human carelessness. Conventionally, such a software quality evaluation method is described in, for example, the Institute of Electronics and Communication Engineers, Journal 7415 Vol.57-DNo.
Static prediction and failure rate transition model- ". Others are, for example, IPSJ 25th (Showa 57).
Yoichi Miyashita entitled "Software Error Dynamics" (1E-1) at the National Convention in the second half of the fiscal year, and Seika Yoshida "Software Quality Goal Management"
(1E-8), respectively. But,
It is said that the cumulative number of bugs extracted by the test is applied to the growth curve to estimate the number of remaining bugs, or the quality target value is managed by making a rough distinction between the development stage and the number of bugs after provision. However, it is insufficient in terms of early detection and accurate grasp of abnormal quality of software.

[Object of the Invention]

An object of the present invention is to provide a software quality automatic evaluation method that automatically evaluates software quality and enables early detection and accurate grasp of abnormal quality.

[Outline of Invention]

Normally, software testing (checking) is repeated several times, but it is considered that the number of items of approximately the same level is randomly confirmed each time. From this, it is considered normal that the transition of the number of bugs extracted in the test repeated several times is of a decay type. This can be explained as follows. As a premise, (1) the total number of statements is M _o , and the number of initial latent bugs is B _o
(B _o <M _o ), the number of statements executed in the k-th test is N _k (N _k <M _o ), the number of potential bugs when executing the k-th test is B _k , and the bug extraction probability is Let P _k .

(2) It is assumed that the probability P _k that an arbitrary statement executed in the k-th test is a bad statement is given by B _k / M _o .

(3) Bad statements will be corrected on the spot, and the number of creation bugs due to the correction will be very small.

Assuming that N≈N _k (k = 1, 2, ...), a bug extraction probability P _k , a bug extraction number N · P _k , and a remaining bug number B _k.
Is It is represented by. The changes in equations (1), (2), and (3) when k = 1, 2, 3, ... Are summarized in the table shown in Fig. 7. Also, the bug extraction probability and the number of remaining bugs are the same. Fig. 8 shows the transition of the graph in Fig. 8, and it can be seen that it decreases exponentially with repeated tests.

The present invention is based on the idea that the software design quality and the test method are normal if the transition of the number of extracted bugs is an attenuation type in the test repeated several times and the other types are abnormal, and the attenuation target The number of cases, the target evaluation value, etc. are given from experience values, etc., and the quality of the software is automatically evaluated by comparing it with the actual value and the transition state of the actual evaluation value itself.

Example of Invention

FIG. 1 shows an overall block diagram of a system configuration to which the present invention is applied. In FIG. 1, a test apparatus 10 is the center of the present invention, and an ordinary minicomputer including a CPU, a ROM, a RAM and the like is used and has a function of automatically performing quality evaluation of software (program). is doing. Software to be tested by the test apparatus 10 is stored in the floppy disk 30. Keyboard 2
0 is used to input a desired operation instruction, data, etc. to the test apparatus 10. The processing result of the test device 10 is output to the display device 40 or the printer device 50.

FIG. 2 shows a processing flow of the present invention in the test apparatus 10. The test device 10 includes a floppy disk 30.
The program under test stored in is read and tested, and the bug is extracted (step 101). In the table prepared in the RAM of the test apparatus 10, the target bug extraction number of each test process unit of each program is pre-read from the keyboard 20, and the test result is
Number of bugs actually extracted in the table (actual number)
Newly read.

Next, the program data to be evaluated is extracted from the extracted bug data (target number, actual number) for each program / test process read in the table of this RAM, and each data of the target and the actual result is tested. Convert to output format and computable format for each process (Step 1
02).

Next, the target number of picked-up bugs and the actual number of picked-up bugs in each test process are compared, and evaluation is performed based on the defective pick-up preliminary result evaluation standard (step 103). Fig. 3 shows an example of the evaluation criteria of defective extraction prediction results. "○" is good, "△" is good.
“X” indicates a defect. It should be noted that a = defective extraction result / target value × 100.

Next, the product of the defect extraction record of each test process up to the process and the quality transition evaluation calculation coefficient is added to each test process unit to calculate the quality transition evaluation value of the process (step 1).
04). The weighting of the quality transition evaluation calculation coefficient is changed for each evaluation process. FIG. 4 shows an example of the quality transition evaluation calculation coefficient.

Next, the evaluation value calculated in step 104 is compared with the transition evaluation value of the immediately preceding test process, and evaluation is performed based on the quality transition evaluation standard (step 105). As shown in FIG. 4, the quality transition evaluation value is calculated such that the larger the reduction rate of the number of extracted bugs in each test process is, the better (lower) value is. FIG. 5 shows an example of quality transition evaluation criteria. Note that e = evaluation value of the process / evaluation value of the immediately preceding process × 100.

Next, the evaluation result is displayed on the display device 40, for example, in the form of a graph (step 106), and if the list is required as a hard copy (step 107),
A hard copy is output using the printer device 50 (step 108).

FIG. 6 shows a specific example of the processing according to the present invention. This is an example in which the number of steps of the program is 25k and the test is 5 steps.

The target value for the number of extracted bugs is 1 per k step based on experience value
Given 2 cases (301 absolute cases), the test process allocation is 163 cases for the first step, 80 cases for the second step, 40 cases for the third step, and 13 cases for the fourth step. The number of cases and fifth process is set to five. Actually, the first process is 87
There are 60 cases in the 2nd step, 34 cases in the 3rd step (0 in 4th step and 5th step), and both design quality and test method are considered to be normal. The second step and the third step are all “Δ” (see FIG. 3). In addition, the target value of the damping coefficient is set to a rate at which the number of extracted bugs is halved in the subsequent process, taking into account the experience value and the expected value (see Fig. 4). The actual evaluation value is the number of extracted bugs and the reduction coefficient. Express as product. When the evaluation step is the second step, the evaluation value is the sum of 87 × 0, 5 (first step) and 60 × 1 (second step), and the transition evaluation is “x” (see FIG. 5). There is. When the evaluation step is the third step, the evaluation value is 87 × 0, 2
5 (1st process) and 60x0, 5 (2nd process) and 34x1
It becomes the sum of (3rd process) and the transition evaluation becomes “◯”, and it is possible to determine that both the design quality and the test method are almost normal at the time of the 3rd process, even considering the above-mentioned preliminary performance evaluation.

According to the example of FIG. 6, the preliminary performance evaluation is “△” in the first process.
Therefore, the responsible design is the extracted bug content and test (check)
We examined the method, and based on the result, set a bug extraction policy in two steps, added test items, man-hours, time, etc.
Make an effort to compensate for the lack of extraction in the process. Even in the second process, the preliminary performance evaluation is "△", so the same examination is performed, but since the transition evaluation is "x", the module with poor design quality is also reviewed and efforts are made to improve quality. In this way, the evaluation of the design quality / testing method is automatically displayed in an easy-to-understand manner, which is effective in promoting the bug extraction in the early testing process.

〔The invention's effect〕

According to the present invention, it is possible to detect abnormalities in the design quality and testing method based on the transition pattern of the number of extracted bugs in the testing process for software, so measures can be taken early in the testing process to prevent low quality shipments and delivery delays. Has an effect on.

[Brief description of drawings]

FIG. 1 is an overall block diagram of a test system to which the present invention is applied, FIG. 2 is a diagram showing an example of a processing flow of the present invention, and FIG.
FIG. 4 is a diagram showing an example of a defect extraction preliminary performance evaluation standard, FIG. 4 is a diagram showing an example of an evaluation value calculation coefficient, FIG. 5 is a diagram showing an example of a quality transition evaluation standard, and FIG. 6 is according to the present invention. FIG. 7, FIG. 7 and FIG. 8 which show a concrete example are diagrams for explaining that the bug extraction transition is of a damping type. 10 ... Test device, 20 ... Keyboard, 30 ... Floppy disk, 40 ... Display device, 50 ... Printer device.

Claims (1)

[Claims] 1. A method for automatically evaluating the quality of software by repeatedly testing the software a plurality of times, wherein the target number of extracted bugs for each test step of the software is stored in a memory, and Of extracting the bugs in each test step and storing the actual number of the extracted bugs in the memory, and comparing the actual number of the extracted bugs and the target number stored in the memory for each test step Then, for each test process unit, the actual number of picked-up bugs in each test process up to the test process is added to each test process by changing the weighting for each test process and adding the test process. And a step of calculating the quality transition evaluation value of the test step, and comparing the quality transition evaluation value of the test process with the quality transition evaluation value of the previous test process. Software Quality automatic evaluation method characterized by a step of evaluating the quality of software in the test process.