JP4847102B2 - Test program generation method for computer program - Google Patents

Description

  The present invention relates to a test program generation method for a computer program that automatically generates a test program using random numbers.

  Conventionally, various methods for generating a test program for a computer program have been proposed. Generally, an inspector or the like manually creates a test program according to the test program syntax, but a method of automatically creating a test program has also been proposed.

  For example, Patent Document 1 describes a test program generator that automatically generates a compiler test program. As shown in FIG. 2, the test program generator 200 uses a set of program descriptions obtained by collecting a plurality of lines of a single sentence created according to the syntax as one element (program cell). The collected program cell information 210 and cell weight information 211 indicating the ease of being selected as a test program from the collected program cells are input to the program cell combiner 230 and generated by the random number generator 220. A compiler test program 240 is automatically generated by combining program cells, which are a collection of selected programs, using random numbers. Cell weight information 211 indicating the ease of being selected as a test program related to a plurality of collected program cells is a program cell in which a defect of a test target program has been found previously, or a program cell that is predicted to be able to find a defect. Given against. The collected program cells are more easily selected as the cell weight indicating the ease of selection when selected by random numbers is heavier.

Patent document 2 describes a language processing program verification method. This technique selects a rule using a random number from a plurality of prepared syntax generation rules, and generates a test program based on the selected rule. Patent Document 3 describes a test program creation support apparatus. This apparatus determines the grammatical coverage of an existing test program, outputs a grammar deviating from the existing test program as a reduced grammar specification, and automatically generates an additional test program based on the reduced grammar specification. Further, Patent Document 4 describes a test program creation support device. This apparatus generates a sentence pattern based on a syntax rule described in Bacchus notation, and manually edits the generated sentence pattern to form a test program.
JP 2004-227129 A JP-A-1-220044 Japanese Patent Laid-Open No. 3-75840 JP-A-6-214775

  For example, in order to identify a compiler defect, it is necessary to perform many syntax combination tests in consideration of syntax combinations, and to create a number of test programs as test data and perform tests. If a test program is created manually by a conventional method, the number of steps is enormous, and there is a possibility that a sufficient test cannot be performed. Therefore, an automatic test program generation method as described above has been proposed.

  However, when a set of program descriptions that consist of multiple lines of a single statement created according to the syntax is used as an element, the syntax combination in the program statement is not determined mechanically by random numbers, but depends on the sense of the creator. Therefore, the syntax combination is not created for the description of complicated conditions beyond the scope of the creator of the program statement.

  In addition, a test program is generated by selecting an element using random numbers for an element for which a program description that has previously found a defect is gathered and an element for which a program description that is expected to be found is gathered. However, in this case, since the weight indicating the ease of selection is fixed to the initially set content, it is always easy to select elements with the same program description regardless of the usage status of the syntax. It becomes difficult to find defects.

  In addition, when generating a test program based on a random generation rule selected from among the syntax generation rules prepared in advance, the syntax generation rule prepared as described above is an area considered by the test program creator. There is a similar problem. Furthermore, when generating a test program by manually selecting the syntax information prepared, or generating a test program with grammar coverage, it is possible for simple contents / items to be tested, but under complicated conditions. It is difficult to detect defects that occur.

  Accordingly, an object of the present invention is to provide a test program generation method for a computer program that automatically generates a test program capable of extracting defects generated under complicated conditions for a test target program.

  The object is to select and confirm one of the test program generation components as a first component using a random number from a group of test program generation components obtained by collecting a plurality of test program generation components of a computer program to be tested. The first step and the step of selecting and confirming one of the n-th components from among a plurality of n-th components constituting the selected (n-1) -th component by using a random number from n to 2 The second step, which is repeated for an integer that increases sequentially for the first time, and when the selected and confirmed n-th component does not have a component constituting it, the information on the selected and confirmed n-th component is selected as a component of the test program And generating a test program based on a plurality of test program components obtained by repeatedly performing the first to third steps. The test program generation method of a computer program that is achieved.

  Here, the selection of the first component in the first step can be performed in consideration of weight information indicating the ease of selection of components provided corresponding to the test program generation components. In addition, the selection of one of the n-th components in the second step can be performed in consideration of weight information indicating the ease of selection of components provided corresponding to the respective n-th components. When a constraint condition in the syntax of the test program occurs during the selection of the n-th component in the second step, weight information indicating the ease of selecting the component can be changed according to the constraint condition. . Further, when the test program is generated, the random number value of the random number is held as reproduction information, and when the test program is reproduced, the random number value held as the reproduction information can be used.

  ADVANTAGE OF THE INVENTION According to this invention, the test program production | generation method of the computer program which produces | generates automatically the test program which can extract the defect which generate | occur | produces on complicated conditions with respect to a test object program can be obtained. That is, the present invention utilizes the fact that the syntax of the test program can be divided into element units, and the test program capable of extracting defects that occur under complicated conditions that are difficult to find with actual measurement data and manually created test programs. Can be generated automatically. Further, at the time of retesting, the debugging efficiency and quality of the test target program can be improved by reproducing the test program using a random number value held as reproduction information. As described above, in the test program generation method of the present invention, the syntax of the test program is divided into element units, and each syntax selected by a random number is embodied as a component with settings corresponding to the development and usage status. A test program corresponding to various tests can be generated.

Hereinafter, an embodiment of a computer program test program generation method according to the present invention will be described with reference to the drawings.
FIG. 1 is a diagram illustrating a configuration example of a test program generator 100 that automatically generates a test program of a computer program. The test program generator 100 shown in the present embodiment is realized by, for example, software running on a personal computer. As shown in the figure, a test program generation component group 101 and a test program generation component realization unit 102 are provided. And a random number machine 103 and a test program output unit 104. The test program generation component realization unit 102 can obtain weight information indicating the ease of selecting a component from the component weight information 105. The test program output unit 104 outputs the generated test program to the test program 106. The random number device 103 exchanges information with the reproduction information 107. Here, the test program is, for example, a compiler test program, but is not limited thereto.

  Here, the program syntax information divided in units of elements constituting the syntax, and the weight indicating the ease of selection used when selecting a part for which the syntax is defined by a random number according to the usage status, are defined. A syntax that can be executed as a program is registered as a component so that statements from a plurality of statements to variable declaration statements can be generated.

  In FIG. 1, a test program generation component group 101 is a group (group) of components in which the syntax of a test program to be generated is divided into units that can be divided and defined as one component. That is, the element that can be divided is each part that constitutes the syntax. Next, an example of this component will be described with reference to FIGS.

  FIG. 4 is a diagram showing a part of a C language syntax example (Bacchus notation) of a test program serving as test data. In this example, the expression syntax in C language is divided into elements and expressed in Bacchus notation. In this figure, “assignment-expression” in the expression, “expression, assignment-expression”, and the like are components constituting the syntax.

  FIG. 5 is a diagram illustrating a part of a C language syntax example (syntax tree) of a test program serving as test data. In this example, the expression syntax in C language is divided into elements and expressed in a tree structure. In this figure, “assignment-expression”, “expression, assignment-expression”, etc. that lead to expression are components that constitute the syntax.

  Both the syntax examples shown in FIGS. 4 and 5 are defined such that expression selects one of the syntaxes of assignment-expression or expression, assignment-expression. However, even though the syntax is simple, there are several situations where it is used, and the meanings are different.

  FIG. 6 is a diagram showing a part of a C language syntax example (Bacchus notation) of a test program serving as test data. In this example, the syntax of “for” in C language is divided into elements and expressed in Bacchus notation. Here, the for syntax is also a part, and the expression group constituting the for syntax is also a part constituting the for syntax. Regarding the for syntax, the first expression, the second expression, and the third expression of the for are all expressed as expressions, but have different meanings such as an assignment statement for initialization, a comparison statement for determination, and an addition / subtraction statement, respectively. Therefore, when automatic generation of test programs is considered, it is necessary to prepare a lot of syntax information suitable for the situation.

  Therefore, it is possible to generate a meaningful test program by dividing the syntax into element units and defining the syntax and processing selected according to the usage status as parts, and without increasing the number of syntax combination information, Since the combination of syntax is determined by selecting parts using random numbers, the number of combinations of syntax will automatically increase if many test programs are generated, and a meaningful test program can be generated without human intervention. And a test program of syntax combination can be generated.

  FIG. 3 is a diagram showing an overall flow when generating a test program to be processed by the test program generator 100. First, in step 301, the presence / absence of a reproduction information instruction is checked. Here, the reproduction information is the reproduction information 107 of the test program output from the random number device 103. When this is “None”, the process proceeds to Step 303 described later. If “Yes”, in step 302, reproduction information setting processing is performed. In the reproduction information setting process 302, by specifying the reproduction information 107 of the test program output from the random number device 103 to the random number device 103 at the initial stage of generating the test program, the corresponding test program can be reproduced. Become. The reproduction information setting process 302 (reproduction of a test program) will be described in detail later with reference to FIG.

  Next, in step 303, the presence / absence of a component weight instruction is inspected. When this is “None”, the process proceeds to Step 305 described later. If “Yes”, in step 304, component weight information setting processing is performed. In the component weight information setting process 302, a weight ratio for each selected component that matches the usage status of the syntax divided into element units defined in the component weight information 105 is set for the corresponding component. The component weight information setting process 304 will be described in detail later with reference to FIG.

  Subsequently, in step 305, test program generation component realization processing is performed. In the component realization process 305, the test program generation component realization unit 102 realizes the component selected by the random number generated by the random number generator 103 from the test program generation component group 101. Here, the realization means that the selected part is given an entity to determine the part. Having an entity means dynamically allocating memory with contents specialized to the usage status of the component. The embodied (determined) part can be further expanded several times to the parts constituting it. While the deployment is possible, the realization process is repeated. The realization process ends when the embodied parts can no longer be developed. Then, the information (type, constant, variable, operator, etc.) of the embodied part (terminal part) at the end is selected and confirmed according to the usage status of the syntax. This end part information constitutes a test program. By performing the above processes one after another, the test program output process 306 outputs a test program. The test program generation component realization process 305 will be described below with reference to FIGS.

  FIG. 7 is a diagram for explaining an example of a test program generation component realization process (when the realization process end flag is OFF). In this example, the syntax expression is registered as a test program generation component 701. The expression component 701 is a weight of the component to be realized that indicates the ease of being selected by the random number in units of the expansion instruction 703 for the expression component 701 given according to the usage status of the syntax for each component 702 to be selected by the random number. A ratio (weight information) is defined.

  As parts of the syntax (implementation target parts) of the expression part 701, there are assignment-expression 704 and Comma-Exp 705 as shown in FIG. These parts to be selected are merely syntax information until they are actualized in the realization process. Further, although Comma-Exp 705 is not a definition that exists in the syntax, “expression, assignment-expression” is usually called a comma operator, and “expression, assignment-expression” is selected as one component unit to be selected. They are collectively used for convenience. Hereinafter, the realization process of the component in which the syntax is divided into element units will be described.

  FIG. 8 is a diagram illustrating an example of a component realization process flow. Hereinafter, the component realization process will be described with reference to FIGS. As shown in FIG. 7, when “for first expression (assignment statement weight)” 707 is given as an expansion instruction to the expression component 701, first, in step 801 of FIG. Check whether a constraint condition has occurred. As a result, if no constraint condition has occurred, the process proceeds to step 803. On the other hand, if a constraint condition occurs, in step 802, the weighting ratio of the component is changed in accordance with the syntax constraint condition. Next, in step 803, the ON / OFF check of the realization process end flag, that is, the determination of the realization process end is performed. The determination of the end of the realization process is to confirm the presence / absence of further selection elements for the parts that have been realized according to the syntax. The realization process end flag 706 defined in the expression part 701 is used. OFF / ON indicates the information.

  In the example of FIG. 7, the realization process end flag 706 is displayed as OFF, which indicates that the realization process has not ended. If it is not the end of the realization process, in step 804 in FIG. 8, the weight ratio set for each realization target part is acquired for each given development instruction. In the case of the example in FIG. 7, the for first formula portion 708 surrounded by a thick dotted line corresponds to this. Subsequently, in step 805, a part to be realized is selected based on the random number generated from the random number generator and the acquired weight ratio, and in step 806, the part is realized by giving an entity. (Block 709 in FIG. 7). In FIG. 7, assignment-expression 704 corresponds to a part selected and embodied by a random number. Then, in step 807, a corresponding deployment instruction is given to the implemented component assignment-expression 704 (arrow 710 in FIG. 7), and component realization processing for determining the subsequent syntax is executed. This realization process is repeated. As a result, the case where the realization end flag is set to “ON” will be described next.

  FIG. 9 is a diagram for explaining an example of a test program generation component realization process (when the realization process end flag is ON). As a result of repetitive component realization processing, if the realization processing flag in the corresponding component is “ON”, this indicates that the corresponding component is a terminal component of the syntax. In FIG. 9, assignment_operator 901 is a part of the assignment operator syntax of the assignment statement. Similarly to the expression 701 described above, the assignment_operator 901 also indicates the ease of being selected by a random number in units of the deployment instruction 903 for the assignment_operator component 901 given according to the usage status of the syntax, depending on the implementation target component 902 selected by the random number. The weight ratio of the parts to be realized is defined. In the assignment operator syntax, the operators that can be used are limited by the usage of the syntax, but there is no syntax expansion after the assignment operator. That is, when the implementation process end flag is “ON”, information (type, constant, variable, operator, etc.) of the implementation component at the end is selected and determined according to the usage status of the syntax, and the subsequent implementation process Without returning to the caller. That is, in step 808 of FIG. 8, the information (type, constant, variable, operator, etc.) of the realization component (terminal component) that is the terminal is selected and confirmed according to the usage status of the syntax. This is a component of the test program. Next, a test program generation method performed based on the above-described implementation process will be described.

  FIG. 12 is a diagram illustrating an example of a test program generation method based on a process for realizing a test program generation component. As shown in the figure, first, a part 109 that is the basis of the syntax is selected and confirmed from the test program generation part group. Next, a part 110 to be implemented with respect to the part 109 is selected from a group of test program generation parts by using a random number to be implemented (determined). At this time, the description will be made hereinafter assuming that the syntax of the assignment statement is determined in the component 110. First, in order to construct an assignment statement, one of the test program generation components is selected as the component 111 from the test program generation component group using a random number and determined. Next, one of the components constituting the component 111 is selected and embodied (determined) to obtain the embodied component 112. Similarly, one of the components constituting the component 112 is selected using a random number and embodied (determined) to obtain the embodied component 113. In the same manner, the embodied parts 114... 11a are obtained sequentially. In this case, the component 11a having no component is the terminal component (final element). The information (type, constant, variable, operator, etc.) of the end part is selected and confirmed according to the syntax usage. In this figure, for example, the information of the terminal part selected and confirmed here is “A”. This is a component of the test program. Subsequently, returning to the original, one of the test program generation components is selected and determined as a component 112 from the test program generation component group using a random number, and the embodied components 122, 123, 124 are the same as described above. ... 12b is obtained. In this case, the component 12b having no component is the end component (final element). The information (type, constant, variable, operator, etc.) of the end part is selected and confirmed according to the syntax usage. In this figure, for example, the information of the terminal part selected and confirmed here is “=”. This is a component of the test program. In this example, the constituent elements of the test program are selected and determined one after another in this way, and the test program is generated based on this. The above process is generalized as follows. That is, a first step of selecting and confirming one of the test program generation parts as a first part from a group of test program generation parts obtained by collecting a plurality of test program generation parts of a computer program to be tested using a random number. And a step of selecting and confirming one of the n-th components from among a plurality of n-th components composing the selected (n-1) -th component using a random number, starting from 2 and increasing in order. A second step that is repeated in step 3 and a third step of selecting and confirming information on the selected and confirmed n-th component as a component of the test program when the selected and confirmed n-th component does not have a component constituting it. And a test program is generated based on the components of a plurality of test programs obtained by repeatedly performing the first to third steps.

  In the above test program generation, the weight ratio change of the selected component will be described. FIG. 9 shows an example of changing the selected component weight ratio due to restrictions. In the component realization process, there are cases where restrictions are imposed on the subsequent syntax determination due to the determined syntax. For example, when the substitution target variable type is determined to be a pointer type (block 905 in FIG. 9), a syntax constraint condition that only a pointer substitution expression (=, + =,-= only possible) occurs for the component assignment_operator 901 ( Block 906 of FIG. An assignment instruction 907 for “assignment formula (all)” has already been instructed in the assignment_operator 901 that has already been realized, and a syntax that does not meet the syntax restrictions is selected with this weight ratio. Therefore, in this case, the process proceeds from step 801 to step 802 in FIG. In this step 802, the weight ratio of the realization target part of the deployment instruction 907 given to the assignment_operator 901 is changed to the weight ratio of the realization part in accordance with the syntax constraint condition (thick dotted line 908 in FIG. 9). As a result, it is possible to eliminate the selection of a component outside the syntax constraint in the subsequent component selection process.

  FIG. 10 is a diagram illustrating an example of changing the selected component weight ratio when the component weight information is designated. As shown in the figure, when the component weight information 1001 is designated (when an instruction is given in step 303 in FIG. 3), the weight of the selected component is changed as follows. That is, in the part weight information 1001, a weight ratio 1004 indicating the ease of selection of the realization target part unit is defined for each corresponding realization part 1002 and a development instruction 1003 given thereto. When the component weight instruction for realization is given at the time of test program generation, the component weight information setting process 1005 for realization (step 304 in FIG. 3) expands the designated component weight information 1001 and the corresponding realization. The expansion instruction partial information 1007 of the converted component expression 1006 is changed to the expanded content 1008. Thus, in the subsequent test program generation component realization process (step 305 in FIG. 3), the weight ratio of the realization target component selected by the random number is changed to the content instructed by the component weight information 1001. As a result, it is easy to select a part having a syntax expected by the user.

  FIG. 11 is a diagram for explaining an example of a random number generator and random number use processing. Hereinafter, the reproduction of the test program will be described with reference to FIG. In the test program generator 100 of FIG. 1, the random number use processing 1110 shown in this figure is performed. The random number use process 1110 executes the random number unit 1101 to generate a random number value 1102 for each process of determining syntax / numerical value such as an implementation process 1103 by component selection, a variable reuse process 1104, a numerical value process 1105, and the like. Based on the value, selection of parts, reuse of variables, and numerical setting are determined.

The random number device 1101 includes a random number reference value setting process 1106, a random number reference value 1107, and a random value generator 1108. The random number reference value setting processing 1106 generates and sets the first random number reference value 1107 used by the random number generator 1108. The reference value to be set first is generated so as to be unique every execution. Further, the content of the random number reference value 1107 is output to the outside as reproduction information 1109. The random number generator 1108 receives the random number generation request from the random number generator 1101, inputs the random number reference value 1107, and outputs the random value 1102. In addition, the random value generated at that time is set as the random number reference value 1107, and is set as the reference value for generating the random number next time. Thus, the next random number value can be generated by automatically setting the random number reference value 1107 of a unique value set every time. Since the random number generator 1101 generates a random number value based on the random number reference value 1107, it is necessary to make the random number reference value 1107 unique every time when the random number device is used. If the random number reference value 1107 set first is the same as the previous one, the random value 1102 generated from the subsequent random number generator 1101 is the same as the previously generated content. Using this, the reproduction information 1109 output in the random number reference value setting process 1106 is set as the random number reference value 1107 via the random number reference value setting process 1106 when the test program is generated, whereby the random number value generated from the random number generator 1101 is changed. Since it is the same as the previous time, the selected part / embodiment process is the same, and the test program generated last time can be reproduced. Further, the reproduction information 1109 can be set not only by the contents output by the random number reference value setting processing 1106 but also by the user when the test program generator is executed.
The content related to the test program generator 100 can be used in other test program generation methods as long as the syntax of the test program can be divided into elements and can be divided into parts. Further, the computer program inspection system can be configured in the same manner as the compiler inspection system disclosed in Patent Document 1, for example.

  The present invention relates to a test program generation method for a computer program that automatically generates a test program using random numbers, and has industrial applicability.

It is a figure which shows the structural example of the test program generator 100 which produces | generates the test program of a computer program automatically. It is a figure which shows an example of the conventional test program production | generation method. It is a figure which shows the whole flow at the time of the test program production | generation processed with a test program generator. It is a figure which shows a part of C language syntax example (Bacchus notation) of the test program used as test data. It is a figure which shows a part of C language syntax example (syntax tree) of the test program used as test data. It is a figure which shows a part of C language syntax example (Bacchus notation) of the test program used as test data. It is a figure for demonstrating an example of the component realization process for test program generation | occurrence | production (when the realization process completion flag is OFF). It is a figure which shows an example of the implementation process flow of components. It is a figure for demonstrating an example of the component realization process for test program generation | occurrence | production (when the realization process completion flag is set to ON). It is a figure which shows an example of the selection component weight ratio change at the time of specifying component weight information. It is a figure for demonstrating an example of a random number machine and a random number utilization process. It is a figure which shows an example of the test program production | generation method by the implementation process of the components for test program production | generation parts.

Explanation of symbols

100 Test Program Generator 101 Test Program Generation Component Group 102 Test Program Generation Component Realization Unit 103 Random Number Unit 104 Test Program Output Unit 105 Component Weight Information 106 Test Program 107 Reproduction Information 301 Reproduction Information Instruction 302 Reproduction Information Setting Process 303 Test Program generation component weight instruction 304 Component weight information setting processing 305 Test program generation component realization processing 306 Test program output processing 701 expression component 702 Expression target component 703 included in the expression Deployment instruction 704 assignment- given to the realized expression expression component 705 Comma-Exp component 706 expression embodying process end flag 707 express ion deployment instructions given to (for the first type)
708 for Selected Formula Weight Ratio in Formula 1 709 Component Selection by Random Number 710 Realization Processing of Components Selected by Random Number 801 Syntax Constraint Condition 802 Processing at Generation of Syntax Constraint 803 Realization Processing End Flag 804 Weight Ratio by Deployment Instruction Acquisition 805 Component selection by random number 806 Realization of selected component 807 Realization processing for component 808 Processing at the end of realization processing 901 assignment_operator component 902 Implementation target component 903 assignment_operator's implementation instruction 904 entition_entrification process End flag 905 Syntax constraint generation condition example 906 Generated syntax constraint condition example 907 given to assignment_operator Expansion instruction 908 Selected part weight ratio for each expansion instruction changed by the syntax constraint condition 1001 Component weight information 1002 Realization part 1003 Realization part-specific expansion instruction 1004 Realization target part weight ratio 1005 Realization part weight information setting processing 1006 Expression component 1007 Expression corresponding deployment instruction 1008 Weight ratio 1110 of the changed realization target component 1110 Random number use processing 1101 Random number 1102 Random value 1103 Realization processing of component using random number 1104 Variable reuse processing 1105 using random number Random number 1106 random number reference value setting process 1107 random number reference value 1108 random number generator 1109 reproduction information

Claims (5)

Programming language syntax for describing a test program generated by dividing the at and the syntax capable division element by element, the test program for generating a collection of components that define it as one test program generation parts A first step of selecting and confirming one of the test program generation parts as a first part from a part group using a random number, and a plurality of nth parts constituting the selected and confirmed (n-1) part. A second step in which a step of selecting and confirming one of the n-th components from a group of test program generation components for the component by using a random number is repeated with an integer that starts from 2 and increases sequentially, and the selection If the confirmed n-th part does not have a part constituting it, the information on the selected and confirmed n-th part is selected and confirmed as a component of the test program. A test program generating method for a computer program, comprising: a third step, wherein a test program is generated based on a plurality of test program components obtained by repeating the first to third steps .
  The selection of the first component in the first step is performed in consideration of weight information indicating ease of selection of a component provided corresponding to each of the test program generation components. A test program generation method for a computer program according to Item 1.   The selection of one of the n-th components in the second step is performed in consideration of weight information indicating ease of selection of components provided corresponding to the respective n-th components. Item 3. A test program generation method for a computer program according to Item 1 or 2.   When a constraint condition in the syntax of the test program occurs when selecting the n-th component in the second step, weight information indicating the ease of selecting the component is changed according to the constraint condition. The test program generation method for a computer program according to claim 3.   The random number value of the random number when the test program is generated is held as reproduction information, and when reproducing the test program, the random number value held as the reproduction information is used. 5. A test program generation method for a computer program according to any one of 4 above.

Images