JP6913071B2 - Test support method, test execution method, test support device and computer program - Google Patents

Description

The present disclosure relates to a test support method, a test execution method, a test support device, and a computer program that realize the efficiency of the test process of a software program.

In the development of many software programs, a quality improvement method called continuous integration is adopted in order to realize quality control of software programs in a short period of time, and creation (build), test and deployment are repeated cyclically. It has been. Non-Patent Document 1 discloses a history in which a program is maintained and updated by a large number of users.

In the test process of continuous improvement, regression testing is performed to test whether other parts are affected by partial changes due to bug fixes, function additions, etc. after the program is released. Is done. Regression testing is important for continuous improvement, but it is time consuming and costly, and is said to cost 50-66% of the lifetime management cost of the program (Non-Patent Document 2).

Many regression tests are formulated by testers on the assumption that the software program in question will be used, and are executed according to the situation. However, it requires man-hours because it is formulated manually. Regression tests have traditionally been devised to improve efficiency, such as creating scripts that perform tests automatically and narrowing down to tests that handle data related to corrections. Among them, Non-Patent Document 3 discloses a method of modeling the dependence of a function or variable included in a software program as an effective graph, determining the importance for an arbitrary input group, and determining the test priority. Has been done. It has been proposed to efficiently select and execute tests in this way.

Software programs that realize analysis of physical phenomena in objects on a computer, so-called CAE (Computer Aided Engineering) programs, are used in various fields such as structural analysis, welding strain analysis, thermo-fluid analysis, and electromagnetic field analysis. .. A software program for physical phenomenon analysis takes a long time to analyze in order to execute a simulation, and reads a file for setting the shape (structure), material, conditions, etc. of the object, which is a prerequisite for analysis, as a specification. , The analysis result is output as a file (Patent Document 1 etc.). At this time, the input / output files are the reserved words (keywords) corresponding to the parameter names, variable names, function names, predetermined values, error numbers, etc. set for each software program for physical phenomenon analysis, and numerical values. It is described by associating logical values and the like (Non-Patent Document 4 and the like).

International Release 2013/084306

“Debian Continuous Integration”, [online], Debian Project, [Search August 10, 2018], Internet <URL: https://ci.debian.net/> R. Pressman, “Software Engineering: A Practitioner's Approach”, McGraw-Hill Education, USA, 2000 Swarnendu Biswas et al., “Regression Test Selection Techniques: A Survey”, Informatica 35, 2011, pp.289-321, Internet <URL: https://users.ices.utexas.edu/~sbiswas/files/papers /informaticasurvey.pdf> “LS-DYNA Manual R10.0-Vol I”, [online], October 16, 2017, Livermore Software Technology Corporation, [Search August 10, 2018], Internet <URL: https://www.dynasupport .com / manuals / ls-dyna-manuals / ls-dyna-manual-r10.0-vol-i>

Software programs for this kind of physical phenomenon analysis have various shapes of objects, and various combinations with materials and boundary conditions. Therefore, as disclosed in Non-Patent Document 1, there are many types of reserved words used to describe the input / output file, and the number of combinations of reserved words for describing the analysis target is enormous. Therefore, the number of test cases for software programs for physical phenomenon analysis is even larger because they basically exist for each reserved word.

By applying the test selection method described above to the software program for physical phenomenon analysis, it is possible to narrow down from a huge number of test cases. However, even if the regression test can be narrowed down to the periphery of the data related to the change, it does not always correspond to the user's needs, and it is not always possible to streamline the test of the part with high needs. In addition, when a configuration that incorporates a software program provided by another person is used and the configuration is unknown or complicated, it is not always possible to select a test based on the design information of the software program.

The purpose of this disclosure is a test support method and test execution that updates the test priority of a software program, reduces the test man-hours, and realizes the automatic operation of the test for continuous improvement that meets the needs of users. To provide methods, test support devices and computer programs.

The test support method according to one aspect of the present disclosure stores inquiries from different users including any one of a plurality of predetermined keywords corresponding to the software program, and the predetermined plurality of inquiries included in the inquiries from the users. One of the keywords of the above is extracted, and a value indicating the priority for the extracted keyword is calculated based on the frequency of appearance of the extracted keyword or the importance of the user who made the inquiry, and is calculated with the plurality of predetermined keywords. It is output in association with the value indicating the priority given.

The test execution method according to one aspect of the present disclosure has a priority calculated based on an inquiry from a user for a plurality of predetermined keywords corresponding to the software program when the completion of updating the software program is detected. The computer references the test cases extracted by, and the computer executes the referenced test cases.

The test support device according to one aspect of the present disclosure includes a storage unit that stores inquiries from different users including any one of a plurality of predetermined keywords corresponding to the software program, and the above-mentioned inquiries from the users. A calculation that calculates a value indicating the priority of the extracted keywords based on the extraction unit that extracts any of a plurality of predetermined keywords, the frequency of appearance of the extracted keywords, or the importance of the user who made the inquiry. It includes a unit and an output unit that outputs the predetermined plurality of keywords in association with a value indicating a calculated priority.

The computer program according to one aspect of the present disclosure records inquiries from different users including any of a plurality of predetermined keywords corresponding to software programs on a computer, and the predetermined inquiries included in the inquiries from the users. One of the plurality of keywords in the above is extracted, and a value indicating the priority for the extracted keyword is calculated based on the frequency of appearance of the extracted keyword or the importance of the user who made the inquiry, and the predetermined plurality of the above. The process of outputting the keyword of is associated with the calculated value indicating the priority is executed.

In one aspect of the present disclosure, any one or more of the keywords (reserved words) corresponding to the parameter name, variable name, function name, predetermined value, error number, etc. set for the predetermined software program are used. Inquiries from different users including are analyzed, and a value indicating the priority of the keyword is calculated by reflecting the importance of the inquiring user as a customer. By calculating the value indicating the priority of the keyword, a test case containing more high-priority keywords or having a higher correlation is selected.

In one aspect of the disclosure, the priority values are calculated by adding the frequency of occurrence or importance calculated for each different period with less influence of those values in the older period.

In one aspect of the present disclosure, the value indicating the priority may be calculated by using any one or all of the number of visits, the number of contract licenses, sales, etc. regarding the target software program as elements. It may be calculated by giving different weighting to each element.

In one aspect of the present disclosure, a learning model for determining whether or not a keyword contained in an inquiry is contextually important in the inquiry is used, and a value indicating priority is calculated using the importance. Will be done.

According to the test support method, the test execution method, the test support device, and the computer program of the present disclosure, the test method can be efficiently selected.

It is a figure which shows the outline of the test support apparatus in this embodiment. It is a block diagram which shows the structure of the test support apparatus. It is a flowchart which shows an example of the process procedure of selection of a test case. It is a flowchart which shows an example of the automatic execution processing procedure of a test. It is a flowchart which shows an example of the priority calculation method. It is explanatory drawing which shows the example of the test case. It is a flowchart which shows the calculation method of the priority in the modification 1. It is a flowchart which shows the calculation method of the priority in the modification 2. It is a figure which shows the content example of a learning model. It is a flowchart which shows the calculation method of the priority in the modification 3.

Specific examples of the test support device and the computer program according to the embodiment of the present disclosure will be described below with reference to the drawings. It should be noted that the present invention is not limited to these examples, and is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

FIG. 1 is a diagram showing an outline of the test support device 1 according to the present embodiment. The test support device 1 supports the development and improvement of the analysis program 1P in cooperation with a support service that provides support to the user regarding the use of the analysis program 1P and a service that manages the sales of the analysis program 1P. The analysis program 1P includes an engine that executes a simulation calculation for analyzing a physical phenomenon. In the simulation calculation, a huge number of steps are executed, the physical phenomena are various, the boundary conditions used in the calculation are also various, and an error is output from the analysis program 1P in the analysis computer that executes the analysis program 1P. It may be terminated or it may be forcibly terminated. When the calculation by the analysis program 1P does not go as intended by the user, the user makes an inquiry to the support service provider. The support service provider accepts inquiries, records the data of the inquiries received, and provides support to the user regarding how to use the analysis program 1P based on the data. The test support device 1 efficiently and automatically executes the test of the analysis program 1P based on the inquiry to the support service provider, identifies bugs, improves the function, and the like.

The analysis program 1P targeted by the test support device 1 of the present embodiment includes an engine for analyzing physical phenomena. This engine for physical phenomenon analysis is described by associating numerical values, logical values, etc. with reserved words (keywords) corresponding to preset parameter names, variable names, function names, predetermined values, error numbers, etc. Read the input file IF and execute the process. The number of reserved words in the engine for physical phenomenon analysis is enormous, and even a simple combination of reserved words is tens of thousands. Furthermore, considering the variation of parameter values (numerical values, logical values, etc.) for the reserved words, the total number is total. Testing is not possible. Therefore, the test support device 1 in the present embodiment selects the input file IF corresponding to the test case to be executed with priority according to the priority of the reserved word determined based on the information regarding the support to the user. The test support device 1 executes an automatic script corresponding to the test case using the selected input file IF, and outputs the execution result to the storage device 3.

The test support device 1 is connected to the server device 2 and the storage device 3 managed by the support service provider. The test support device 1, the server device 2, and the storage device 3 are connected to the same local network LN. The server device 2 can send and receive information to and from the communication terminal device 5 used by the user via the local network LN and the public network N.

The server device 2 includes a mail server function for sending and receiving e-mails with a support service administrator as a sender or a receiver, and functions as an interface for inquiries from users. The server device 2 can store an inquiry mail about the analysis program 1P targeted by the support service in the storage device 3. The email contains time information such as transmission time or reception time. In addition, the server device 2 has a Web server function and receives inquiries about the analysis program 1P on the Web page. The server device 2 stores the content of the inquiry for each user in the storage device 3 in association with the time information when the inquiry is received. Further, the server device 2 stores the information about the support regarding the analysis program 1P in the storage device 3. In the description of the first embodiment, the server device 2 is simplified as one server computer having the functions of a mail server and a Web server. The server device 2 may be distributed to a plurality of server computers and storage media for processing, or is one of a plurality of server computers (instances) virtually generated by one large computer. You may.

The storage device 3 is shared by the support service provider. The storage device 3 stores the reserved words of the analysis program 1P. The reserved word is a keyword corresponding to the parameter name, variable name, function name, predetermined value, error number, etc. set for the analysis program 1P. The storage device 3 stores a plurality of patterns of reserved words and test input file IFs in which numerical values, logical values, etc. for the reserved words are described corresponding to the test cases of the analysis program 1P according to the function of the analysis program 1P. .. The input file IF for the test may be automatically created by the test support device 1 or another device, or may be created by the operator using the test support device 1. ..

The test support device 1 stores the target analysis program 1P, and can automatically execute the test based on the analysis program 1P. The test support device 1 is used by the operator of the support service or the developer of the analysis program 1P, and as described above, the test support device 1 uses the information stored in the storage device 3 to give priority to the test (input corresponding to the test case). Create information for selecting file IF). The test support device 1 automatically tests the analysis program 1P by the test script corresponding to the test case using the created information. The process of creating information for selecting a test and the process of executing a test of the analysis program 1P may be performed by different devices.

The network N includes a public network. The public network is the so-called Internet. The public network may include an access point AP. The communication terminal device 5 used by the user can transmit information to the server device 2 via the network N.

FIG. 2 is a block diagram showing the configuration of the test support device 1. The server device 2 to which the test support device 1 is connected has the functions of the mail server 21 and the Web server 22. The server device 2 stores the mail data received from the information terminal device used by the user by the mail server 21 and the contents of the inquiry obtained by the Web server 22 in the storage device 3 as inquiry data.

The storage device 3 referred to by the test support device 1 stores information shared by the support service provider using a large-capacity storage medium such as an SSD or a hard disk. The storage device 3 will be described as one storage device 3 for the sake of simplicity, but the storage media of the plurality of storage devices 3 are virtually treated as one large-capacity storage medium. good. The storage device 3 can read and write from the test support device 1 and the server device 2 that can communicate with the local network LN.

As described above, the storage device 3 has one or more reserved words and reserved words set for the analysis program 1P, and input file IFs corresponding to a plurality of test cases used according to the design of the analysis program 1P. It is remembered. The storage device 3 further stores a test execution history including success or failure of a test executed for any one of a plurality of stored test cases.

The storage device 3 stores inquiry data including an inquiry mail from a user regarding the analysis program 1P by the server device 2 or a text describing the contents of the inquiry received by the Web server 22. The storage device 3 stores the correspondence (visit, visit) history for each user by the support service. Further, the storage device 3 stores contract information (number of contracts, contract history, contract version, etc.) for each user for the analysis program 1P obtained from the sales management system for the analysis program 1P.

The test support device 1 includes a control unit 10, a storage unit 11, a display unit 12, an operation unit 13, and a communication unit 14. Although the test support device 1 will be described below as one computer device for ease of explanation, it may be a virtual machine using the hardware resources of one or a plurality of computer devices.

The control unit 10 is a processor using a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit), and includes a built-in volatile memory, a clock, and the like. The control unit 10 can execute each process based on the analysis program 1P stored in the storage unit 11. Therefore, use a processor with specifications that can withstand the computational load. Then, the control unit 10 executes a process of giving priority to the reserved words to be prioritized based on the test selection support program 100P stored in the storage unit 11.

The storage unit 11 stores the analysis program 1P using a non-volatile storage medium such as an SSD (Solid State Drive) or a hard disk. The test selection support program 100P is stored in the storage unit 11. The storage unit 11 stores the test script 1S corresponding to the test case of the analysis program 1P. The test script 1S is stored together with the test case in association with the test case identification information in the storage device 3, and may be duplicated in the storage unit 11 by the control unit 10. The storage unit 11 stores the data referred to by the control unit 10.

The display unit 12 includes a display device such as a liquid crystal panel or an organic EL display. The operation unit 13 is an interface that accepts user operations, and includes input / output interfaces such as a keyboard and a mouse. The operation unit 13 may include a touch panel device with a built-in display.

The communication unit 14 is a communication module that realizes a connection to the server device 2 and the storage device 3 via the local network LN. Specifically, the communication unit 14 is, for example, a network card, and performs communication via a network LN by wire.

In this way, the process of selecting the test case of the analysis program 1P by the test support device 1 will be described with specific examples. The operator uses the operation unit 13 of the test support device 1 to set the test selection support program 100P and the execution schedule of the automatic test. For example, the control unit 10 executes the following processing at a periodic timing such as once a day, once a week, or once a month, or at a timing when the operator gives an execution instruction. For each execution timing, the inquiry data read from the storage device 3, the response history, and the test execution history are limited to the inquiry data received during the target period from the previous execution time to the current execution time, and for each target period. Perform processing.

FIG. 3 is a flowchart showing an example of a test case selection processing procedure. The control unit 10 of the test support device 1 reads the inquiry data of the target period from the storage device 3 (step S1), and uses the reserved words included in the inquiry data as user identification information (user) that identifies the user who sends the e-mail or Web inquiry. It is extracted in association with the account) (step S2). The control unit 10 reads out the correspondence history and the contract information from the storage device 3 for each user identification information (step S3).

The control unit 10 calculates a value indicating the priority of each of the reserved words extracted in step S2, reflecting the importance of the mail or the user who made the inquiry in the frequency of appearance in the inquiry of the reserved word (step S4). ). The method of calculating the value indicating the priority will be described later.

The control unit 10 uses the priority value calculated for each reserved word, and obtains the comprehensive evaluation value for each test input file IF (test case) described using the reserved word. The calculation is performed in consideration of the test execution history (step S5). In step S5, the control unit 10 changes the comprehensive evaluation value depending on the success or failure of the test. For example, the control unit 10 may be varied so that the comprehensive evaluation value of the unexecuted test cases becomes high. Since the regression test is so important that the test that was successful last time fails, the control unit 10 evaluates the overall evaluation value of the test that was successful in the previous execution, for example. To fluctuate.

The control unit 10 extracts input file IFs (test cases) for a predetermined number of times that can be performed for each period in descending order of the comprehensive evaluation value calculated in step S5 (step S6). The control unit 10 outputs test case identification information for identifying each of the extracted input file IFs (step S7), and ends the process. The output in step S7 may be either an output to a file or an output to the display unit 12.

It is preferable that the test support device 1 reads out the test script 1S based on the output test case identification information and automatically executes it, following the selection processing procedure shown in the flowchart of FIG. FIG. 4 is a flowchart showing an example of the test automatic execution processing procedure. The test support device 1 may execute the following process immediately after the selection process shown in the flowchart of FIG. Preferably, when it is detected that the update (build) in the continuous improvement of the analysis program 1P is completed, the following processing may be executed to execute the regression test.

The control unit 10 refers to the extracted test case identification information output by the procedure shown in the flowchart of FIG. 3 in order of priority (step S11). The control unit 10 uses the input file IF corresponding to the referenced test case identification information and executes the test based on the corresponding test script 1S (step S12).

The control unit 10 outputs the success or failure of the test, which is the execution result of the test, as the test execution history (step S13). The test execution history is stored in the storage device 3 as a file, or may be output as a screen on the display unit 12 of the test support device 1.

The control unit 10 sequentially executes the processes of steps S11 to S13 for all or a part of the extracted test cases.

Next, the method of calculating the priority for the test case will be described in detail. In the priority calculation method of the present embodiment, the items of the elements that contribute to the priority are set in advance and are stored in the storage unit 11. Further, a weight for the priority is set for each element, and the weight is similarly stored in the storage unit 11. The control unit 10 of the test support device 1 reads elements from the storage device 3 for each reserved word or each user based on these items, assigns a set weight, and calculates a value indicating the priority. In an example described below, the control unit 10 sets the "frequency of appearance" of the reserved word, the importance of the user of the sender of the mail or inquiry using the reserved word, "number of responses (number of visits, visits)", and ". Use the number of contracted licenses. In addition, regarding the importance of the user, "contract version (premium or not, etc.)", "number of emails or inquiries", etc. may be used.

FIG. 5 is a flowchart showing an example of a priority calculation method. The control unit 10 calculates the number of occurrences of each reserved word, and extracts reserved words whose number of appearances is equal to or greater than a predetermined number (at least "1") (step S401). The control unit 10 specifies the user identification information of the user who is the sender of the mail or the inquiry including the reserved word for each of the extracted reserved words (step S402).

Next, the control unit 10 selects one user identification information from all the contracted users (step S403). The control unit 10 reads out the “number of contract licenses” for the corresponding user or an organization such as a company to which the user belongs from the storage device 3 for the selected user identification information (step S404). The control unit 10 reads out the “number of correspondences” in the target period for the selected user identification information from the “correspondence history” of the storage device 3 (step S405).

The control unit 10 determines whether or not the user identification information of all the contracted users has been selected (step S406), and if it is determined that the user identification information of all the users has not been selected (S406: NO), the process is stepped. Return to S403 and select the next user identification information. When it is determined in step S406 that the user identification information of all users has been selected (S406: YES), the control unit 10 proceeds to the next step S407.

The control unit 10 selects one of the reserved words extracted in step S401 (step S407), and calculates the frequency of appearance of the selected reserved words (ratio of the number of occurrences to the number of appearances of all reserved words) (step S408). ). The control unit 10 reads out the frequency calculated in step S408, the "number of contract licenses" read in step S404 for the user identification information specified in step S402 for the selected reserved word, and the "number of correspondences" calculated in step S405. A value indicating the priority is calculated based on the ratio of the above to the whole and the weight value set for each (step S409). As a result, the priority is calculated.

When it is determined whether or not the control unit 10 has selected and executed the processing for all the reserved words extracted in step S401 (step S410), and it is determined that all the reserved words have not been executed (S410: NO). , The control unit 10 returns the process to step S407.

When it is determined in step S410 that the processing has been executed for all the reserved words (S410: YES), the calculation of the value indicating the priority is finished, and the control unit 10 performs the processing in step S5 shown in the flowchart of FIG. return.

As described above, the flowchart of FIG. 5 is only an example of calculating the priority, and if the test case including an important reserved word can be prioritized and automatically executed in the inquiry from the customer to be prioritized, another The method may be used. In a simple example, a hash table or the like is used, and information is stored in the storage device 3 by a data structure for adding a user who has made an inquiry using a reserved word to a hash list using the reserved word as a key. In this case, processing such as user search (S402), reserved word selection, and search becomes unnecessary.

The calculation method shown in the flowchart of FIG. 5 will be described with reference to specific examples. In a specific example, for three reserved words "* KEYWORD1", "* KEYWORD2", and "* KEYWORD3" whose number of appearances in the target period is "1" or more, "USER1", "USER2", and "USER3" Calculate the priority that reflects the importance of the user who has the user identification information (user name).

The number of appearances of each reserved word is as follows, and the weight for "appearance frequency" is 0.5.
"* KEYWORD1": 3 times (1 time from "USER1", 2 times from "USER2")
"* KEYWORD2": 1 time (1 time from "USER3")
"* KEYWORD3": 7 times (2 times from "USER1", 5 times from "USER3")

The "number of correspondences" of each user is as follows, and the weight with respect to the ratio of the "number of correspondences" is 0.25.
"USER1": 1 time, "USER2": 5 times, "USER3": 0 times

The "number of contracted licenses" is as follows, and the weight with respect to the ratio of the "number of contracted licenses" is 0.25.
"USER1": 1, "USER2": 1, "USER3": 2

Here, the weights for each element are designed so that the total is "1".

The control unit 10 calculates the appearance frequency of the first selected “* KEYWORD1” as follows (S408).
"* KEYWORD1": 3 / (3 + 1 + 7) = 3/11

In step S410, the control unit 10 calculates the ratio of the "number of responses" for the "USER1" and "USER2" that have made an e-mail or inquiry including the selected "* KEYWORD1" as follows.
"USER1": 1 / (1 + 5 + 0) = 1/6
"USER2": 5 / (1 + 5 + 0) = 5/6
The ratio of "number of contract licenses" is calculated as follows.
"USER1": 1 / (1 + 1 + 2) = 1/4
"USER2": 1 / (1 + 1 + 2) = 1/4

In step S410, the control unit 10 calculates a value indicating the priority of the selected “* KEYWORD1” for the “USER1” and “USER2” identified as using the selected “* KEYWORD1” as follows.
Value indicating priority = {Percentage of "number of responses" for "USER1" * 0.25
+ Ratio of the same "number of contract licenses" * 0.25
＋ "Appearance frequency" * 0.5}
+ {Percentage of "number of responses" for "USER2" * 0.25
+ Ratio of the same "number of contract licenses" * 0.25
＋ "Appearance frequency" * 0.5}
= {(1/6) * 0.25 + (1/4) * 0.25 + (3/11) * 0.5}
+ {(5/6) * 0.25 + (1/4) * 0.25 + (3/11) * 0.5}
= {0.0416 + 0.0625 + 0.13636}
+ {0.2083 + 0.0625 + 0.13636}
= 0.64762

Next, the control unit 10 calculates the appearance frequency of the second selected “* KEYWORD2” as follows (S408).
"* KEYWORD2": 1 / (3 + 1 + 7) = 1/11

In step S410, the control unit 10 calculates a value indicating the priority of the selected “* KEYWORD2” for the “USER3” identified as using the selected “* KEYWORD2” as follows.
Value indicating priority = {Percentage of "number of responses" for "USER3" * 0.25
+ Ratio of the same "number of contract licenses" * 0.25
＋ "Appearance frequency" * 0.5}
= {(0) * 0.25 + (2/4) * 0.25 + (1/11) * 0.5}
= {0 + 0.125 + 0.04545}
= 0.17045

Next, the control unit 10 calculates the appearance frequency of the third selected “* KEYWORD3” as follows (S408).
"* KEYWORD1": 7 / (3 + 1 + 7) = 7/11
In step S410, the control unit 10 calculates, for the selected “* KEYWORD3”, a value indicating the priority of “USER1” and “USER3” identified as using the selected “* KEYWORD3” as follows.
Value indicating priority = {Percentage of "number of responses" for "USER1" * 0.25
+ Ratio of the same "number of contract licenses" * 0.25
＋ "Appearance frequency" * 0.5}
+ {Percentage of "number of responses" for "USER3" * 0.25
+ Ratio of the same "number of contract licenses" * 0.25
＋ "Appearance frequency" * 0.5}
= {(1/6) * 0.25 + (1/4) * 0.25 + (7/11) * 0.5}
+ {(0) * 0.25 + (2/4) * 0.25 + (7/11) * 0.5}
= {0.0416 + 0.0625 + 0.31818}
+ {0 + 0.125 + 0.31818}
= 0.86546

Therefore, in step S4 in the flowchart of FIG. 3, a value indicating the priority is calculated for each reserved word as follows.
"* KEYWORD1": 0.64762
"* KEYWORD2": 0.17045
"* KEYWORD3": 0.86546

FIG. 6 is an explanatory diagram showing an example of a test case. As shown in FIG. 6, the input file IF of the test case "001" includes "* KEYWORD1" twice and "* KEYWORD2" once, and the input file IF of "002" contains "* KEYWORD2". * KEYWORD3 "is included once. In step S5 in the flowchart of FIG. 3, the control unit 10 calculates the overall evaluation value of each of the input file IFs of "001" and "002" as follows (S5). In this example, the control unit 10 adds only the types of reserved words included in the input file IF, and if they overlap, the control unit 10 does not add the total evaluation value. Moreover, it is assumed that the test execution history has the same conditions.
Comprehensive evaluation value of "001" = value indicating the priority of "* KEYWORD1"
+ A value indicating the priority of "* KEYWORD2"
= 0.64762 + 0.17045
= 0.81807
Comprehensive evaluation value of "002" = value indicating the priority of "* KEYWORD3"
= 0.64762 + 0.17045
= 0.86546

In this way, the control unit 10 calculates the overall evaluation value higher for the test case "002" than for the test case "001" (S5).

In this case, in the case where only one test case is selected, the control unit 10 extracts the test case of "002" (S6).

In the above example, the comprehensive evaluation value was calculated by the sum of the values indicating the priority. However, not limited to this, the correlation between the distribution of reserved words included in the test case and the reserved word group with high priority is calculated, and the value indicating the calculated high correlation is used as the comprehensive evaluation value. The test case may be selected according to the above.

As shown in the above example, in the present embodiment, the function related to the changed part and the test case using the reserved word are not always preferentially selected as the test case. In the present embodiment, the test case relating to the reserved word, which is frequently inquired by the user who should be prioritized, is preferentially executed. As a result, it is possible to realize a quality improvement that satisfies the user by performing a test that is beneficial to the user, rather than selecting a test case and executing the test based on the design or implementation. Further, in the case of a program incorporating another engine such as the analysis program 1P in the present embodiment, the test case can be selected regardless of the implementation and the changed part, so that the engine design and the implementation status are unknown. Even so, you can run valid tests. In this way, the test support device 1 of the present disclosure makes it possible to realize continuous improvement in line with the actual situation of use of the software program.

(Modification example 1)
The method of calculating the value indicating the priority of the reserved word is not limited to the procedure shown in the flowchart of FIG. 5, and can be carried out by various methods using elements that contribute to the priority. FIG. 7 is a flowchart showing a method of calculating the priority in the first modification. The control unit 10 selects one user identification information from all the contracted users (step S421). The control unit 10 extracts reserved words included in an email or inquiry from the user for the selected user identification information (step S422). The control unit 10 calculates the frequency of appearance in the mail or inquiry from the user selected for the extracted reserved word (step S423).

Next, with respect to the selected user identification information, the control unit 10 reads out the “number of contract licenses” for the corresponding user or an organization such as a company to which the user belongs from the storage device 3 (step S424). The control unit 10 reads out the “number of correspondences” in the target period for the selected user identification information from the “correspondence history” of the storage device 3 (step S425). Of course, the information regarding the importance of the user used in the calculation is not limited to these.

The control unit 10 has the appearance frequency calculated in step S423 for each reserved word used by the user extracted in step S422, the "number of contract licenses" read in step S424, and the "number of correspondences" calculated in step S425. , A value related to the priority for each reserved word for each user is calculated based on the weight value set for each (step S426). The control unit 10 temporarily stores the value related to the priority of each reserved word for each user in association with the user identification information (step S427).

The control unit 10 determines whether or not the user identification information of all the contracted users has been selected (step S428), and if it is determined that all the users have not been selected (S428: NO), the process is stepped. Return to S421 and select the next user identification information. When it is determined in step S428 that the user identification information of all users has been selected (S428: YES), the control unit 10 proceeds to the next step S429.

The control unit 10 selects 1 reserved word from the reserved words extracted in step S421 for all users, that is, a group of reserved words extracted at least once among all reserved words as having a predetermined number of occurrences or more. One is selected (step S429). In step S429, the reserved words may be narrowed down by the number of occurrences more than the predetermined number of times.

With respect to the reserved words selected in step S429, the user who sent the e-mail or inquiry including the reserved words is specified (step S430), and the reserved words for each user temporarily stored in step S427 for the specified users are used. The values related to the priority are totaled, and the value indicating the priority is calculated (step S431).

The control unit 10 determines whether or not all the reserved words extracted for all users have been selected (step S432), and if it is determined that all the reserved words have not been selected (S432: NO), the process Is returned to step S429.

When it is determined in step S432 that all the reserved words have been selected (S432: YES), the control unit 10 returns the process to step S5 shown in the flowchart of FIG.

An example of calculating the priority shown in the flowchart of FIG. 7 will be specifically described. The appearance conditions for the three reserved words "* KEYWORD1", "* KEYWORD2", and "* KEYWORD3" are the same as in the above example.

In the example shown in the flowchart of FIG. 7, the value indicating the priority for each reserved word can be expressed as follows.
Value indicating the priority of the reserved word = Sum of the values of each element indicating the priority calculated for each user who used the reserved word

For example, for the value related to the priority of "KEYWORD1", add up the values related to the priority calculated for each user as shown below.
Value related to priority of "KEYWORD1" = {"frequency of appearance" * 0.5 + ratio of "number of responses" * 0.25 for "USER1"
+ Ratio of "number of contract licenses" * 0.25}
+ About "USER2"{"Appearancefrequency" * 0.5 + "Number of responses" * 0.25
+ Ratio of "number of contract licenses" * 0.25}
= {(1/3) * 0.5 + (1/6) * 0.25 + (1/4) * 0.25}
+ {(2/2) * 0.5 + (5/6) * 0.25 + (1/4) * 0.25}
= 0.27077 + 0.7708
= 1.04157

Here, since the "appearance frequency" is the frequency of appearance during an email or inquiry for each user, for example, for "USER1", "* KEYWORD1" is once and "* KEYWORD3" is twice, so the frequency of appearance is as follows. Is.
"* KEYWORD1": 1 / (1 + 0 + 2) = 1/3
"* KEYWORD3": 2 / (1 + 0 + 2) = 2/3
Is.

Similarly, the value related to the priority of "KEYWORD2" = {"frequency of appearance" * 0.5 + ratio of "number of responses" * 0.25 for "USER3"
+ Ratio of "number of contract licenses" * 0.25}
= {(1/6) * 0.5 + (0) * 0.25 + (2/4) * 0.25}
= {0.0833 + 0 + 0.125}
= 0.2083

Similarly, the value related to the priority of "KEYWORD3" = {"frequency of appearance" * 0.5 + ratio of "number of responses" * 0.25 for "USER1"
+ Ratio of "number of contract licenses" * 0.25}
+ About "USER3"{"Appearancefrequency" * 0.5 + "Number of responses" * 0.25
+ Ratio of "number of contract licenses" * 0.25}
= {(2/3) * 0.5 + (1/6) * 0.25 + (1/4) * 0.25}
+ {(5/6) * 0.5 + (0) * 0.25 + (2/4) * 0.25}
= =
= {Ratio of "appearance frequency" * 0.5 + "number of responses" * 0.25
+ Ratio of "number of contract licenses" * 0.25}
= {(5/6) * 0.5 + (0) * 0.25 + (2/4) * 0.25}
= {0.43743 + 0.54167}
= 0.9791

When calculated by the processing procedure shown in the flowchart of FIG. 7, the priority of "KEYWORD1" is higher than that of "KEYWORD3".

(Modification 2)
In the above example, the priority is calculated by using the mail, inquiry, and response history in the target period, but in the following, the history of other target periods is also used for calculation. FIG. 8 is a flowchart showing a method of calculating the priority in the second modification. Among the processing procedures shown in the flowchart of FIG. 8, the same step numbers as those of the processing shown in the flowchart of FIG. 5 in the first embodiment are assigned the same step numbers, and detailed description thereof will be omitted.

When the control unit 10 executes the processes up to steps S401 to S410, the control unit 10 stores it in the storage unit 11 as a value indicating the priority at the time point t in association with the time information (step S411). The control unit 10 reads out a value stored in the storage unit 11 indicating the priority of each reserved word calculated up to the previous time (step S412).

The control unit 10 uses the value indicating the priority stored in step S411 and the value indicating the priority calculated up to the previous time read in step S412, and multiplies the discount rate η to give priority in consideration of the past value. A value indicating the degree is calculated (step S413). The control unit 10 returns the process to step S5 in the flowchart of FIG. 3 using the value calculated in step S413.

Assuming that the value indicating the priority at the time point t is Wt, the value M indicating the priority calculated in step S413 is calculated by the following discount sum reward.
M = Wt + η ^{Wt-1 + η 2} Wt-2 + ... + η ^k Wt-k + ...
Note that Wt-k is a value W indicating the priority calculated k times before.

This makes it possible to calculate the priority for reserved words that reflect past evaluations, and makes it possible to select test cases more efficiently.

(Modification example 3)
The test support device 1 of the present embodiment basically calculates the priority of reserved words based on the frequency of inquiries, emails, and the like. However, even if reserved words are frequently included, the way the reserved words are used in inquiries and emails is different. For example, depending on the context, there should be a difference such as positive (thank you for support service) or negative (report of defect). In the third embodiment, the appearance frequency of the reserved word reflects the weight based on the context for the reserved word.

For the weight based on the context, a learning model of natural language processing for the reserved words included in the mail data and the inquiry data stored in the storage device 3 is created. The created learning model is stored in the storage device 3, and when a new target period arrives, new mail data and inquiry data during the target period are given to the learning model and derived.

FIG. 9 is a diagram showing an example of the contents of the learning model 3L. The learning model 3L includes an input layer 301 for inputting text data which is mail data or inquiry data, an output layer 302 for outputting a set of reserved words included in the text data and importance based on the context of the reserved words. Includes the intermediate layer 303. The intermediate layer 303 learns from the teacher data (data set) given whether the reserved words included in the input text data have positive content, negative content, or intermediate content. It's done. The output layer 302 may output only the importance, not the reserved words. As shown in FIG. 9, the nodes of the output layer 302 correspond to each reserved word, and the weight for the reserved word corresponds to the importance.

FIG. 10 is a flowchart showing a method of calculating the priority in the modified example 3. Among the processing procedures shown in the flowchart of FIG. 10, the procedures common to the processing procedures shown in the flowchart of FIG. 7 are given the same step numbers and detailed description thereof will be omitted.

Regarding the user identification information selected in step S421, the control unit 10 sequentially reads out the text data of the mail or inquiry from the user from the storage device 3 (step S441) and gives it to the input layer 301 of the learning model 3L (step S442). .. The control unit 10 temporarily stores a set of reserved words output from the output layer 302 of the learning model 3L and the importance of the reserved words (step S443). The processing of steps S442 and S443 is executed for the mail or inquiry from the selected user in the target period, whereby the reserved word included in the mail or inquiry from the user of the selected user identification information is extracted.

The control unit 10 uses the importance output by the learning model 3L for each mail or inquiry for each of the extracted reserved words, and calculates a value indicating the priority based on the accumulation, average, etc. of the importance (step S444). ).

After that, the control unit 10 reads out the "number of contracted licenses" of the selected user (S424), reads out the "number of correspondences" from the storage device 3 (S425), and the like, and the importance of the user selected from these information. Is calculated and stored (step S445).

The control unit 10 identifies a user who used the reserved word for each selected reserved word (S430), and integrates a value indicating the priority calculated in step S444 for the reserved word for each specified user. In this way, the comprehensive evaluation value is calculated (step S446). In step S446, when the total evaluation value is integrated, the control unit 10 gives a weight of the importance of the user to the priority calculated for each user.

In this way, even if the importance and severity differ depending on how the reserved words are used (context), it is possible to appropriately select test cases that should be prioritized by reflecting this information.

In the above-described embodiment, test cases are selected using reserved words (keywords) set for the physical phenomenon analysis engine included in the analysis program 1P. However, the target program of the test selection support program 100P shown in the present embodiment is not limited to the analysis program 1P for physical phenomenon analysis. Since the test selection support program 100P determines the priority for the reserved words related to the test to be executed with priority, how to use the determined priority to extract the test case (corresponding input file IF). The key can be changed depending on the design.

1 Test support device 10 Control unit 1P analysis program 100P Test selection support program 3 Storage device 3L Learning model

Claims (11)

The computer remembers inquiries from different users that include one of a number of predetermined keywords that correspond to the software program.
The computer extracts one of the predetermined keywords included in the inquiry from the user, and the computer extracts one of the predetermined keywords.
Based on the frequency of appearance of the extracted keywords and the importance of the user who made the inquiry, the computer calculates a value indicating the priority for the extracted keywords.
The computer outputs the predetermined plurality of keywords in association with each other and a value indicating the calculated priority .
In the calculation of the value indicating the priority,
The computer
An input layer that accepts inquiries from users, an output layer that outputs the importance of any one of the predetermined plurality of keywords included in the inquiry, and the importance specified from the inquiry and the context of the inquiry. Using a learning model that includes an intermediate layer trained based on teacher data that includes
The query is given to the input layer of the learning model,
A test support method for calculating a value indicating a priority for an extracted keyword by using the importance for any one of the predetermined plurality of keywords output from the output layer in response to the inquiry. The computer
For the test case for the software program including any one of the predetermined plurality of keywords, the evaluation value is calculated using the value indicating the priority calculated for the plurality of keywords.
The test support method according to claim 1, wherein the test cases are extracted in the order of high evaluation based on the calculated evaluation value. Execution history is stored in association with the test case.
The test support method according to claim 2, wherein the evaluation value is calculated by a computer based on the execution history in a test case that has not been executed. The inquiry is stored in association with the time information.
The value indicating the priority is any of claims 1 to 3, which is calculated by a computer by adding the frequency or importance of the occurrence for each different period by multiplying the component of the older period by a low coefficient. The test support method described in item 1. The number of visits for each of the different users regarding the software program is stored.
The test support method according to any one of claims 1 to 4, wherein the importance of the user is calculated by a computer using the number of visits as an element. The number of contract licenses of the software program is stored for each of the different users.
The test support method according to any one of claims 1 to 4, wherein the importance of the user is calculated by a computer using the number of contracted licenses as an element. The sales based on the contract of the software program are stored for each of the different users.
The test support method according to any one of claims 1 to 4, wherein the importance of the user is calculated by a computer using the sales as an element. The value indicating the priority according to any one of claims 1 to 7, which is calculated by a computer by a computer with different weighting for each of a plurality of items that contribute to the frequency of appearance and the importance of the user. Test support method. When the completion of updating the software program is detected, the priority calculated based on the frequency of appearance of the keyword and the importance of the user for a plurality of predetermined keywords corresponding to the software program based on an inquiry from the user. The computer refers to the test cases extracted by the degree,
The computer is a test execution method for executing the referenced test case.
The priority is
An input layer that accepts inquiries from users, an output layer that outputs the importance of any one of the predetermined plurality of keywords included in the inquiry, and the importance specified from the inquiry and the context of the inquiry. The computer uses a learning model including an intermediate layer trained based on teacher data including and.
When a new inquiry from the user is received, any one of the predetermined plurality of keywords included in the inquiry from the user is extracted.
The query is given to the input layer of the learning model,
A test execution method calculated by the computer as a value indicating the priority for the extracted keyword by using the importance for any one of the predetermined plurality of keywords output from the output layer in response to the inquiry. .. A storage unit that stores inquiries from different users including any of a plurality of predetermined keywords corresponding to the software program, and a storage unit.
An extraction unit that extracts any of the predetermined plurality of keywords included in the inquiry from the user, and an extraction unit.
A calculation unit that calculates a value indicating the priority of the extracted keywords based on the frequency of appearance of the extracted keywords and the importance of the user who made the inquiry.
It is provided with an output unit that outputs a plurality of predetermined keywords in association with a value indicating a calculated priority .
The calculation unit
An input layer that accepts inquiries from users, an output layer that outputs the importance of any one of the predetermined plurality of keywords included in the inquiry, and the importance specified from the inquiry and the context of the inquiry. Using a learning model that includes an intermediate layer trained based on teacher data that includes
The query is given to the input layer of the learning model,
A test support device that calculates a value indicating the priority for the extracted keyword by using the importance for any one of the predetermined plurality of keywords output from the output layer in response to the inquiry. On the computer
Record inquiries from different users, including any of a number of given keywords that correspond to a software program,
Extract any one of the predetermined plurality of keywords included in the inquiry from the user, and
Based on the frequency of appearance of the extracted keywords and the importance of the user who made the inquiry, a value indicating the priority for the extracted keywords was calculated.
A process of associating a plurality of predetermined keywords with a value indicating a calculated priority and outputting them is executed .
In the calculation of the value indicating the priority,
On the computer
An input layer that accepts inquiries from users, an output layer that outputs the importance of any one of the predetermined plurality of keywords included in the inquiry, and the importance specified from the inquiry and the context of the inquiry. Using a learning model that includes an intermediate layer trained based on teacher data that includes
The query is given to the input layer of the learning model,
A computer program for executing a process that calculates a value indicating the priority for the extracted keyword by using the importance for any one of the predetermined plurality of keywords output from the output layer in response to the inquiry.

Images