JP7629880B2 - Program modification support system and program modification support method - Google Patents

Description

The present invention relates to a program correction support system and a program correction support method.

There has been a technique for identifying a location containing a defect and generating a patch for the location in the debugging work in the unit test of a program. For example, Patent Document 1 is one such technique. Patent Document 1 describes the following: identifying a defective location in a software program, generating a possible repair at the defective location based on a repair candidate, automatically generating a first test to test the possible repair, generating a first query for a first test oracle based on the first test, obtaining a response to the first query from a human, generating a first human-provided test oracle based on the first query and the obtained response to the first query, adding the first human-provided test oracle to the first test to generate an extended first test, augmenting a test suite to include the extended first test, and testing the possible repair using the augmented test suite.

JP 2019-29015 A

The above-mentioned Patent Document 1 discloses only rearranging arguments as a method for generating test case inputs. This method may not necessarily generate test cases that are effective for verifying patch candidates.

For example, if the test case for the function median(n1, n2, n3) that finds the median is originally "the output of median(2, 6, 8) is 6," it is not possible to generate median(8, 6, 2) or median(6, 8, 2) as new inputs. Also, for example, if a defect occurs when a negative number is entered as an argument in the original source code, the defect will be overlooked when identifying the defective location. Furthermore, if a similar defect occurs in the generated patch, it will be overlooked when verifying the patch candidate, and the patch with the defect will be output.

On the other hand, brute-force or random generation of test case inputs generates many unnecessary test cases and increases the effort required of developers to set the outputs (expectations).

The present invention aims to provide a technology that can generate test cases that are effective for verifying patch candidates without generating unnecessary test cases.

The program correction support system according to the present invention is a program correction support system that supports program correction using a computer having a calculation unit and a storage unit, and is configured as a program correction support system characterized in that the calculation unit analyzes source code, obtains input value conditions and outputs of the source code obtained by the analysis, and expands the test cases based on correction time information indicating candidates for correction of the input value conditions and outputs and test addition criteria for increasing the number of test cases that test the source code using test code.

According to the present invention, it is possible to generate test cases that are effective for verifying patch candidates without generating unnecessary test cases.

FIG. 2 is a diagram illustrating an example of a hardware configuration of the program modification support system according to the present embodiment. 2 is a diagram illustrating an example of a functional configuration of the program correction device illustrated in FIG. 1 . FIG. 13 is a diagram illustrating an example of source code. FIG. 13 illustrates an example of a test code. FIG. 11 is a diagram showing an example of a defective part candidate; FIG. 13 is a diagram illustrating an example of patch candidates. FIG. 13 is a diagram showing an example of a test addition criterion. FIG. 13 is a diagram illustrating an example of generated test code. FIG. 13 is a diagram showing an example of a correction result. FIG. 13 is a diagram illustrating an example of a flowchart showing a processing procedure of a program correction process performed in the present embodiment. FIG. 11 is a diagram illustrating an example of a flowchart showing a processing procedure of the test generation process performed in step S1003 shown in FIG. 10;

The following describes an embodiment of the present invention with reference to the drawings. The following description and drawings are examples for explaining the present invention, and some parts have been omitted or simplified as appropriate for clarity of explanation. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

The position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings.

In the following explanation, various types of information may be explained using expressions such as "database," "table," and "list," but the various types of information may be expressed in data structures other than these. To indicate independence from data structure, "XX table," "XX list," and the like may be referred to as "XX information." When explaining identification information, when expressions such as "identification information," "identifier," "name," "ID," and "number" are used, these are interchangeable.

When there are multiple components with the same or similar functions, they may be described using the same reference numerals with different subscripts. However, when there is no need to distinguish between these multiple components, the subscripts may be omitted.

In addition, the following description may describe processing performed by executing a program, but the program is executed by a processor (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)) to perform a specified process using storage resources (e.g., memory) and/or interface devices (e.g., communication ports) as appropriate, so the subject of the processing may be the processor. Similarly, the subject of the processing performed by executing a program may be a controller, device, system, computer, or node having a processor. The subject of the processing performed by executing a program may be a calculation unit, and may include a dedicated circuit (e.g., an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit)) that performs a specific process.

The program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server may include a processor and a storage resource that stores the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. Also, in the following description, two or more programs may be realized as one program, and one program may be realized as two or more programs.

The following describes the configuration of one embodiment of the program correction support system and program correction support method according to this embodiment.

1 is a diagram showing an example of the hardware configuration of a program modification support system in this embodiment. In the following, an example is shown in which the program modification support system is configured by a program modification device 100, which is a single computer, but all or part of these functions may be distributed and provided in one or more computers, such as a cloud, and similar functions may be realized by communicating with each other via a network. Specific processes performed by each unit constituting the program modification device 100 will be described later using a flowchart.
As shown in FIG. 1, the program modification device 100 can be realized by a general computer having a processor 110 such as a CPU, a main storage device 120 such as a memory, an auxiliary storage device 130 such as a HDD (Hard Disk Drive), an input device 140 such as a scanner, keyboard, or mouse, an output device 150 such as a display, and a communication device 160 such as a NIC (Network Interface Card) for connecting to a communication network.

Figure 2 is a diagram showing an example of the functional configuration of the program correction device 100 shown in Figure 1. As shown in Figure 2, the program correction device 100 according to this embodiment is configured to have a defect location identification unit 210, a patch candidate generation unit 220, a test generation unit 230, a patch candidate verification unit 240, and an information storage unit 290. The information storage unit 290 also stores source code 291, test code 292, defect location candidates 293, patch candidates 294, test addition criteria 295, generated test code 296, and correction results 297.

The various data stored in the program modification device 100 or used in processing (e.g., source code 291, test code 292, defect location candidate 293, patch candidate 294, test addition criteria 295, generated test code 296, modification result 297) can be realized by the processor 110 reading them from the main memory device 120 or the auxiliary memory device 130 and using them. Also, each functional unit of the program modification device 100 (e.g., defect location identification unit 210, patch candidate generation unit 220, test generation unit 230, patch candidate verification unit 240) can be realized by the processor 110 loading a specific program stored in the auxiliary memory device 130 into the main memory device 120 and executing it.

The above-mentioned specific program may be stored (downloaded) into the auxiliary external storage device 130 from a network via the communication device 160 or from a storage medium such as a USB (Universal Serial Bus), and then loaded onto the main storage device 120 and executed by the processor 110, or may be loaded directly onto the main storage device 120 and executed by the processor 110.

The defect identification unit 210 uses the source code 291 and the test code 292 for testing the source code 291 to identify candidates for defect locations in the source code 291.

The patch candidate generation unit 220 generates patch candidates that correct candidate defect locations in the source code based on a specified correction pattern.

The test generation unit 230 generates test cases for the source code based on test criteria related to verification of candidate defect locations in the source code and/or candidate patches, as well as test addition criteria described below.

Figure 3 is a diagram showing an example of source code 291. Source code 291 is code that includes a defect. As will be described in detail later, Figure 3 shows, for example, that source code 291 includes a defect location 301 in which "true" is written where "false" should actually be written.

Figure 4 is a diagram showing an example of test code 292. Test code 292 is code for identifying defective parts of source code 291. Figure 4 shows, for example, that when a numeric value "5" is input into source code 291, the expected value is "false" (test1), and when a numeric value "20" is input into source code 291, the expected value is "false" (test2).

Figure 5 is a diagram showing an example of defect location candidate 293. Defect location candidate 293 is a table that stores defect location candidate included in source code 291. As shown in Figure 5, defect location candidate 293 stores an ID for identifying a defect location and a defect location candidate identified by the ID in association with each other. Figure 5 shows, for example, that the defect location candidate identified by ID "1" is "line 2."

Figure 6 is a diagram showing an example of a patch candidate 294. Patch candidate 294 is code obtained by modifying source code 291 using a specified modification pattern. As shown in Figure 6, patch candidate 294 is stored in association with an ID for identifying the patch candidate and the patch candidate identified by that ID. In Figure 6, for example, the patch candidate identified by ID "1" indicates that it is code obtained by modifying "x>1" on the second line of source code 291 to "x>=1".

Figure 7 is a diagram showing an example of test addition criteria 295. Test addition criteria 295 is a table that defines criteria for adding test cases. As shown in Figure 7, test addition criteria 295 stores an ID for identifying the criteria to be added and the test addition criteria identified by that ID in association with each other. Although not specifically illustrated in this example, it is assumed that the test criteria before being added are defined by various conventionally known methods.

In FIG. 7, for example, the additional criterion for the test identified by ID "1" is "input value that increases the coverage of the defective part candidate". Patent Document 1 describes rearranging arguments as a method for generating input for a test case, but does not disclose the criteria for testing, and source code defects cannot always be tested appropriately. In this example, in addition to "input value that increases the coverage of the defective part candidate", "boundary value of the conditional expression of the patch candidate" is exemplified as an additional criterion for testing. As will be described later in detail, in the former example, test code is generated for a defective (or highly likely) method or instruction block as a result of identifying the defective part, so that the coverage of the part (coverage rate of the process executed in the test, conditional branch, and execution path) is improved, thereby improving the verification accuracy of the patch candidate. In the latter example, test code is generated based on the patch candidate output as a result of patch candidate generation, so that, for example, boundary values can be generated to improve the verification accuracy of the patch candidate. As an example, if if(x>0) should be corrected to if(x>1), but if(x>1) is generated as a patch candidate, adding a test with the boundary value x=1 as input can eliminate the incorrect patch candidate.

Figure 8 is a diagram showing an example of generated test code 296. Generated test code 296 is code in which test code according to the criteria defined in test addition criteria 295 has been added to test code 292. As shown in Figure 8, generated test code 296 indicates that new code shown as test3, test4, and test5 has been added to test1 and test2 included in test code 292 shown in Figure 4.

Figure 9 is a diagram showing an example of the correction result 297. The correction result 297 is code that shows the result of correcting the source code 291. As shown in Figure 9, the correction result 297 shows that the source code 291 shown in Figure 2 has been corrected with the code with ID "3" of the patch candidate 294 shown in Figure 6.

Figure 10 is an example of a flowchart showing the processing steps of the program correction process performed in this embodiment.

In S1010, the defect identification unit 210 applies the test code 292 shown in FIG. 4 to the source code 291 shown in FIG. 3 to identify a defect in the source code 291. For example, the defect identification unit 210 calculates the likelihood that each line in the source code contains a defect, and identifies a line for which the calculation result is equal to or greater than a predetermined threshold as a defect. The defect identification unit 210 outputs the identified defect as a defect candidate 293 shown in FIG. 5.

In S1020, the patch candidate generation unit 220 outputs source code obtained by correcting the output defect location in the source code 291 using a predetermined correction pattern as a patch candidate 294. In this way, the patch candidate generation unit 220 generates multiple correction patch proposals that pass the test by rewriting lines that are likely to contain defects. The above-mentioned predetermined correction pattern may be a pattern that describes the correction content, such as inserting missing statements, changing the instance generation method, changing a conditional expression, etc., that is determined in advance.

In S1030, the test generation unit 230 generates test code for each of the output patch candidates based on the test addition criteria 295. The specific processing of S1003 will be described later with reference to FIG. 11.

At S1040, the patch candidate verification unit 240 executes the generated test code and outputs patch candidates that pass various predefined tests as the final patch proposal.

Figure 11 is an example of a flowchart showing the processing steps of the test generation process performed in S1003 shown in Figure 10.

In S1110, the test generation unit 230 analyzes the source code 291 before modification and obtains pairs of input value conditions and output values. For example, the test generation unit 230 analyzes the source code 291 shown in FIG. 3 and obtains the input conditions "x>1" and "x≦1" and the corresponding outputs "true" and "false", respectively. This information can be realized using a technique such as symbolic execution.

In S1120, the test generation unit 230 obtains an input value and an expected value that executes an instruction or condition that is not executed in the test code before it is added. The expected value may be the output value of the source code before modification. For example, in the test code 292 shown in FIG. 4, a test case that satisfies "x≦1" in the source code 291 is not described. Therefore, the test generation unit 230 obtains an input value of "X=0" that satisfies the test case that satisfies "x≦1". As shown in S1120, the test generation unit 230 reads the test addition criteria 295 shown in FIG. 7, and obtains the "input value that increases the coverage of the defective part candidate" that corresponds to ID "1".

In S1130, the test generation unit 230 analyzes each patch candidate and obtains the boundary value of the input condition. The boundary value can be obtained using a technique such as symbolic execution. For example, the test generation unit 230 obtains "x=1" as the boundary value for the patch candidate with ID "1" shown in FIG. 6. The test generation unit 230 also obtains "x=10" as the boundary value for the patch candidates with IDs "2" and "3". As shown in S1130, the test generation unit 230 reads the test addition criteria 295 shown in FIG. 7 and obtains the "boundary value of the conditional expression of the patch candidate" corresponding to ID "2".

In S1140, the test generation unit 230 presents the acquired test input values to the developer and acquires the correct expected values from the developer. For example, the test generation unit 230 displays the acquired boundary values "x=1" or "x=10" on a display, which is an example of the output device 150. Also, for example, the test generation unit 230 acquires from the developer the expected values "true" or "false" input from a keyboard, which is an example of the input device 140.

In S1150, the test generation unit 230 generates test code with the input values acquired in S1120 and S1130, and the expected value acquired in S1140 as test cases. For example, the test generation unit 230 adds code 801 including the input value "X=0" and expected value "false" acquired in S1120 to the generated test code 296. The test generation unit 230 also adds code 802 including the input value "X=1" and expected value "false" acquired in S1130, and code 803 including the input value "X=10" and expected value "true" to the generated test code 296.

In this embodiment, in a program correction support system that supports program correction using a computer having a calculation unit (e.g., a CPU) and a storage unit (e.g., a memory), the calculation unit analyzes source code (e.g., source code 291), obtains input value conditions and outputs (e.g., a set of "x>1" and "true", "x≦1" and "false") of the source code obtained by the analysis, and expands the test cases based on correction time information (e.g., defect location candidate 293, patch candidate 294) indicating candidates for correction of the input value conditions and outputs, and test addition criteria (e.g., test addition criteria 295) for increasing the number of test cases that test the source code using test code (e.g., test code 292). In this way, by expanding the test cases based on the correction time information and the test addition criteria, incorrect patch candidates can be excluded with high accuracy in patch candidate verification. In addition, by expanding only the test cases based on the correction time information and the test addition criteria, the developer's effort in setting expected values can be reduced.

The modification time information also includes a candidate for the defective part of the source code (e.g., "line 2" of source code 291 shown as defective part candidate 293), and the calculation unit generates a test case including an input that increases the coverage of the defective part (e.g., test case "X=0" that satisfies "x≦1" in source code 291), and outputs test code including the test case. Therefore, in patch candidate verification, incorrect patch candidates can be excluded with high accuracy while taking into account the coverage of the defective part.

The modification time information also includes a patch candidate for the source code (e.g., patch candidate 294 obtained by modifying source code 291 with a predetermined modification pattern), and the calculation unit generates a test case including an input of a boundary value for the patch candidate (e.g., boundary value "x=1" for patch candidate with ID "1" of patch candidate 294), and outputs test code including the test case. Therefore, in patch candidate verification, incorrect patch candidates can be excluded with high accuracy while taking into account the boundary values for the patch candidates.

The calculation unit also generates a test input value (e.g., the input value of "X=0"), obtains an expected value for the generated input value from outside the program correction support system (e.g., obtains the expected value "true" input by the developer), generates a test case including the input value and the expected value, and generates test code including the test case. Therefore, it is possible to generate test cases as expected (designed) by the developer, and as a result, the quality of the generated test cases can be improved.

In this embodiment, the acquired boundary value is displayed on a display, which is an example of the output device 150, or the expected value is acquired as input from a keyboard, which is an example of the input device 140. However, if the input device 140 includes an audio input device such as a microphone, or the output device 150 includes an audio output device such as a speaker, it is also possible to use other types of interface that can be recognized by the user, such as outputting the boundary value by voice through the device, or accepting input of the expected value.

The present invention is not limited to the above-described embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention, or multiple components disclosed in the above-described embodiment can be appropriately combined.

100 Program correction device 210 Defective part identification unit 220 Patch candidate generation unit 230 Test generation unit 240 Patch candidate verification unit 290 Information storage unit 291 Source code 292 Test code 293 Defective part candidate 294 Patch candidate 295 Test addition criteria 296 Generated test code 297 Correction result

Claims (6)

A program correction support system that supports program correction by a computer having a calculation unit and a storage unit,
The calculation unit is
Analyzing a source code, and obtaining input value conditions and outputs of the source code obtained by the analysis;
In a case where the test cases are expanded based on modification time information indicating candidates for modification of the input value conditions and the output and test addition criteria for increasing the number of test cases for testing the source code by the test code,
The modification time information includes candidates for defect locations in the source code,
the calculation unit generates a test case including an input that increases coverage of the defective portion, the input improving coverage of a process, a conditional branch, and an execution path executed in a test, and outputs a test code including the test case;
A program correction support system comprising: The modification time information includes a patch candidate for the source code,
the calculation unit generates a test case including an input of a boundary value in the patch candidate, and outputs a test code including the test case.
2. The program correction support system according to claim 1. the calculation unit generates a test input value, acquires an expected value for the generated input value from outside the program correction support system, generates a test case including the input value and the expected value, and generates a test code including the test case;
2. The program correction support system according to claim 1. A program modification support method performed in a program modification support system that supports program modification using a computer having a calculation unit and a storage unit, comprising:
Analyzing a source code, and obtaining input value conditions and outputs of the source code obtained by the analysis;
In a case where the test cases are expanded based on modification time information indicating candidates for modification of the input value conditions and the output and test addition criteria for increasing the number of test cases for testing the source code by the test code,
The modification time information includes candidates for defect locations in the source code,
the calculation unit generates a test case including an input that increases coverage of the defective portion, the input improving coverage of a process, a conditional branch, and an execution path executed in a test, and outputs a test code including the test case;
23. A program modification support method comprising: The modification time information includes a patch candidate for the source code,
generating a test case including an input of a boundary value in the patch candidate, and outputting a test code including the test case;
5. The program correction support method according to claim 4 . generating input values for a test, acquiring expected values for the generated input values from outside the program correction support system, generating test cases including the input values and the expected values, and generating test code including the test cases;
5. The program correction support method according to claim 4 .

Images