JP7736635B2 - Test result visualization file generation device, test result visualization file generation method, and test result visualization file generation program - Google Patents

Description

The present invention relates to a test result visualization file generation device, a test result visualization file generation method, and a test result visualization file generation program.

Patent Document 1 discloses technology for improving the efficiency of log verification during system operation tests (see paragraph 0011 of Patent Document 1).

JP 2016-71400 A

When an application is developed, it must be tested to see if it operates correctly on the screen. However, conventional methods, including the technology described in Patent Document 1, have not been able to efficiently check the results of these operational tests.

In other words, in the past, it was not possible to view the results of an operational test, the screen showing the results of the operational test, the execution history when the operational test was conducted, or detailed configuration information for the application that was the subject of the operational test in a list. As a result, if a problem arose during an operational test, the person in charge had to manually search for the execution history and detailed configuration information, which required a great deal of time and effort.

The present invention was made in consideration of the above-mentioned problems, and aims to provide a test result visualization file generation device, test result visualization file generation method, and test result visualization file generation program that can visualize the results of tests that automatically run a developed application on a screen and generate visualization files that make it easy to identify where problems occur.

In order to solve the above-mentioned problems and achieve the objectives, the test result visualization file generation device of the present invention is equipped with a control unit that can generate a visualization file that visualizes the results of a test that automatically executes a developed application on a screen and makes it easy to identify the location of a problem.The control unit compares a test image, which is an image of the screen resulting from the test, with a reference image, which is a reference image to be compared, and determines whether the result is no difference, a difference, or incomparable.The control unit embeds the comparison results, which are different or incomparable, and the file names of the test images that are determined to be different or incomparable, in the visualization file based on comparison result data that includes the comparison results linked to the file names of the test images. The system comprises a comparison result embedding means, an image embedding means for embedding the test images determined to be different or incomparable into the visualization file, an execution history embedding means for identifying screen operation instructions including an operation instruction for taking a screenshot of the test images determined to be different or incomparable from a screen operation instruction table containing screen operation instructions consisting of blocks of operation instructions that are instructions for operations automatically executed on the screen during the test, and embedding the contents of execution history data, which is data including an execution history when operations are executed in accordance with the identified screen operation instructions, into the visualization file, and a first link embedding means for embedding a link for checking detailed setting information for the application that was the subject of the test into the visualization file based on the screen operation instructions identified by the execution history embedding means.

Furthermore, in the test result visualization file generation device according to the present invention, the execution history embedding means embeds the contents of the execution history data into the visualization file in a manner that makes it possible to identify execution history corresponding to an operation command commanding taking a screenshot of the test image that has been determined to be different or incomparable.

Furthermore, in the test result visualization file generation device according to the present invention, the image embedding means embeds the test image determined to have a difference, the reference image with which the test image was compared, and an image highlighting the differences between the two images into the visualization file.

In the test result visualization file generation device according to the present invention, the control unit further comprises: processing error information embedding means for, when an execution history indicating a processing error exists in execution history data, which is data including an execution history when operations are performed in accordance with the screen operation commands in the screen operation command table, embedding, in the visualization file, information for identifying the fact that a processing error occurred, information for identifying a screen operation command including an operation command that commanded an operation corresponding to the execution history indicating a processing error, and an execution history when operations were performed in accordance with a screen operation command including an operation command that commanded an operation corresponding to the execution history indicating a processing error, based on the execution history data and the screen operation command table; processing error cause embedding means for embedding, in the visualization file, as the cause of the processing error, an operation command that commanded an operation corresponding to the execution history indicating a processing error; and second link embedding means for embedding, in the visualization file, a link for checking detailed configuration information for the application that was the target of the test, based on a screen operation command including an operation command that commanded an operation corresponding to the execution history indicating a processing error.

Furthermore, in the test result visualization file generation device according to the present invention, the processing error information embedding means embeds, into the visualization file, an execution history when operations are performed in accordance with screen operation commands that include operation commands that command operations corresponding to execution history indicating that a processing error has occurred, in a manner that enables the execution history indicating that a processing error has occurred to be identified.

Furthermore, the test result visualization file generation device according to the present invention is characterized in that the visualization file is a text file.

Furthermore, the test result visualization file generation device according to the present invention is characterized in that the text-format file is an HTML-format file.

The test result visualization file generation method according to the present invention is executed by an information processing device having a control unit, and is capable of generating a visualization file that visualizes the results of a test in which a developed application is automatically executed on a screen and makes it easy to identify where a problem occurs. The test result visualization file generation method executed by the control unit compares a test image, which is an image of the screen resulting from the test, with a reference image, which is a reference image to be compared, and determines whether the test image corresponds to no difference, a difference, or incomparability. Based on the comparison result data linked to the filenames of the test images, the method embeds the comparison results of differences or incomparability and the filenames of the test images determined to be different or incomparable in the visualization file. a step, an image embedding step of embedding the test image determined to be different or incomparable in the visualization file, an execution history embedding step of identifying screen operation instructions including an operation instruction to take a screenshot of the test image determined to be different or incomparable from a screen operation instruction table containing screen operation instructions consisting of blocks of operation instructions that are instructions for operations that are automatically executed on the screen during the test, and embedding the contents of execution history data, which is data including an execution history when operations are executed in accordance with the identified screen operation instructions, into the visualization file, and a first link embedding step of embedding a link for checking detailed setting information for the application that was the subject of the test in the visualization file based on the screen operation instructions identified in the execution history embedding step.

The test result visualization file generation program of the present invention is a program for executing on an information processing device having a control unit, which is capable of visualizing the results of a test in which a developed application is automatically executed on a screen and generating a visualization file that makes it easy to identify where a problem occurs. The program is executed by the control unit. The program includes comparison results, which are results of comparing a test image, which is an image of the screen resulting from the test, with a reference image, which is a reference image to be compared, and determining whether the test image corresponds to no difference, a difference, or incomparability, and includes comparison results linked to the filenames of the test images. The program embeds the comparison results of differences or incomparability and the filenames of the test images determined to be different or incomparable in the visualization file. The method includes a comparison result embedding step, an image embedding step of embedding the test image determined to be different or incomparable into the visualization file, an execution history embedding step of identifying screen operation instructions including an operation instruction to take a screenshot of the test image determined to be different or incomparable from a screen operation instruction table containing screen operation instructions consisting of blocks of operation instructions that are instructions for operations automatically executed on the screen during the test, and embedding the contents of execution history data, which is data including an execution history when operations are executed in accordance with the identified screen operation instructions, into the visualization file, and a first link embedding step of embedding a link for checking detailed setting information for the application that was the subject of the test into the visualization file based on the screen operation instructions identified in the execution history embedding step.

This invention has the advantage of being able to visualize the results of tests that automatically run a developed application on a screen, and to generate a visualization file that makes it easy to identify where problems occur.

FIG. 1 is a block diagram showing an example of the configuration of a test result visualization file generation device. FIG. 2 is a diagram showing an example of an image of an outline of processing in this embodiment. FIG. 3 is a diagram showing an example of an HTML file generated in this embodiment. FIG. 4 is a diagram showing an example of comparison result.txt (comparison result data). FIG. 5 is a diagram illustrating an example of an execution setting file. FIG. 6 is a diagram showing an example of a folder structure of evidence. FIG. 7 is a diagram showing an example of a flowchart of processing when an error occurs in the image comparison (when comparison is impossible) or when there is a difference in the image comparison (when there is a difference). FIG. 8 is a diagram showing an example of a flowchart of processing when an error is found in the execution log of the screen operation program (when a processing error is found). FIG. 9 is a diagram showing an example of an HTML file that is generated when there is a difference in the image comparison (when there is a difference). FIG. 10 is a diagram showing an example of an HTML file that is generated when an error occurs in the image comparison (when comparison is not possible). FIG. 11 is a diagram showing an example of an HTML file that is generated when an error occurs in the execution log of the screen operation program (when a processing error occurs). FIG. 12 is a diagram showing an example of embedding a title when an error occurs in image comparison. FIG. 13 is a diagram showing an example of embedding an image when an error occurs in the image comparison. FIG. 14 is a diagram showing an example of embedding log information and highlighting target processes when an error occurs in image comparison. FIG. 15 is a diagram showing an example of embedding link information when an error occurs in image comparison. FIG. 16 is a diagram showing an example of a screen that is displayed when the character strings "UI definition information" and "execution environment" in the HTML file are clicked. FIG. 17 is a diagram showing an example of comparison result data used for embedding a title when there is a difference in the image comparison. FIG. 18 is a diagram showing an example of embedding an image when there is a difference in the image comparison. FIG. 19 is a diagram showing an example of embedding a title when a processing error occurs. FIG. 20 is a diagram showing an example of embedding detailed information (operation instructions) when a processing error occurs. FIG. 21 is a diagram showing an example of embedding link information when a processing error occurs.

Embodiments of a test result visualization file generation device, a test result visualization file generation method, and a test result visualization file generation program according to the present invention are described in detail below with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments.

[1. Overview, etc.]
In this section, the outline, background, and effects of this embodiment will be described.

[1-1. Overview of this embodiment]
This embodiment is intended to improve the efficiency of checking the results of automated testing of Web applications. In recent years, the speed of evolution of operating systems, browsers, etc. has accelerated, and in order to respond to changes in the external environment, there is a demand for improving the efficiency of development and testing of business applications.

For this reason, it has become essential to automate testing of developed web applications using web browsers.

In this embodiment, by comparing test evidence in HTML format, it is possible to visualize the differences between normal cases and cases where a problem occurs during testing, as shown in Figure 2. This improves the reliability of test evidence and enables early identification of problem areas.

[1-2. Background of this embodiment]
Previously, the person in charge had to visually compare the output images (evidence) of screenshots to find problematic execution results. This resulted in a heavy workload for the person in charge. Another problem was that simply comparing the output images of screenshots did not allow for a clear understanding of the execution conditions of the test cases and automatic execution definitions, which meant that it took a long time to identify the problem areas.

[1-3. Effects of this embodiment]
Therefore, in this embodiment, for example, the following 1 to 6 are made possible.

1. By saving screenshots of reference images and test images for each test case in a specified hierarchy and format, it is now possible to manage evidence while maintaining correspondence.

2. It is now possible to generate comparison HTML that allows you to easily check the correspondence between the evidence reference image and the test image after the test is executed. This makes it possible to see a list of images by test case or progress.

3. It is now possible to generate HTML for checking differences that extracts only the differences between the reference image and test image of the evidence, and to display the following summaries (A) and (B) in a list.
(A) Test case and automatic execution command information (B) Image showing the difference between the reference image and the test image

4. It is now possible to store historical information from when automatic tests were executed on the comparison difference summary list HTML. This makes it possible to instantly understand which test cases and which automatic execution processes have differences.

5. The HTML summary list of comparison differences now includes links to the UI definitions and execution definitions for automated tests that have differences. This makes it possible to smoothly check test cases based on the information on where the differences occur.

6. In short, in this embodiment, in automatic regression testing, evidence management is possible that maintains the correspondence with files corresponding to test cases, and files for comparison and reference (e.g., HTML) can be output. This makes it possible to quickly identify problem areas from the results of test evidence.

The specific configuration and operation are explained below.

2. Configuration
An example of the configuration of a test result visualization file generation device 100 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the test result visualization file generation device 100.

The test result visualization file generation device 100 is a commercially available desktop personal computer. Note that the test result visualization file generation device 100 is not limited to stationary information processing devices such as desktop personal computers, but may also be portable information processing devices such as commercially available notebook personal computers, PDAs (Personal Digital Assistants), smartphones, and tablet personal computers.

The test result visualization file generation device 100 comprises a control unit 102, a communication interface unit 104, a memory unit 106, and an input/output interface unit 108. Each unit of the test result visualization file generation device 100 is connected to each other so that they can communicate with each other via any communication path.

The communication interface unit 104 communicatively connects the test result visualization file generation device 100 to the network 300 via a communication device such as a router and a wired or wireless communication line such as a dedicated line. The communication interface unit 104 has the function of communicating data with other devices via a communication line. Here, the network 300 has the function of connecting the test result visualization file generation device 100 and the server 200 so that they can communicate with each other, and is, for example, the Internet or a LAN (Local Area Network). Note that data such as various masters described below may be stored in the server 200, for example.

An input device 112 and an output device 114 are connected to the input/output interface unit 108. The output device 114 may be a monitor (including a home television), a speaker, or a printer. The input device 112 may be a keyboard, a mouse, a microphone, or a monitor that functions as a pointing device in conjunction with a mouse. Note that, below, the output device 114 may be referred to as the monitor 114, and the input device 112 may be referred to as the keyboard 112 or the mouse 112.

The storage unit 106 stores various databases, tables, files, and the like. The storage unit 106 stores computer programs that work in conjunction with the OS (Operating System) to issue commands to the CPU (Central Processing Unit) to perform various processes. The storage unit 106 can be, for example, a memory device such as RAM (Random Access Memory) or ROM (Read Only Memory), a fixed disk device such as a hard disk, a flexible disk, or an optical disk.

The memory unit 106 includes, for example, comparison result data 106a, a screen operation command table 106b, execution history data 106c, and an HTML file 106d as a visualization file.

The test result visualization file generation device 100 according to this embodiment can generate a visualization file that visualizes the results of a test that automatically executes a developed application on a screen and also makes it easy to identify where a problem has occurred. The visualization file is, for example, a text file, and preferably an HTML file (HTML file 106d).

The comparison result data 106a is data that includes the comparison result, which is the result of comparing a test image, which is an image of the screen resulting from the test, with a reference image, which is a reference image to be compared, and determining whether the result is no difference, a difference, or incomparable, and is linked to the file name of the test image. The comparison result data 106a is, for example, a text file.

For example, the comparison result data 106a shown in Figure 4 includes the result "Valid" indicating that the test image file name "X-XXXX-003.png" is not different from the reference image, the result "Difference" indicating that the test image file name "X-XXXX-001.png" is different from the reference image, and the result "Error" indicating that the test image file name "X-XXXX-002.png" is not comparable to the reference image.

The screen operation command table 106b is a table containing screen operation commands consisting of blocks of operation commands, which are commands for operations to be automatically executed on the screen during the test. As shown in FIG. 14, for example, the screen operation command table 106b contains, in association with each other, the screen operation command and definition identification data (definition ID and definition name) for defining and identifying the screen operation command.

For example, the screen operation command table 106b shown in FIG. 14 includes a screen operation command (definition Id = X002) consisting of a group of three operation commands. As shown in FIG. 14, the three operation commands are: an operation command of type "launch menu" and target "X-XXXX-002", an operation command of type "set value", target "customer CD", and value "1", and an operation command of type "screenshot", file name "X-XXXX-002.png", and confirmation content "Is 1 displayed for customer CD_To?".

Execution history data 106c is data that includes the execution history when operations are performed in accordance with the screen operation commands in the screen operation command table 106b. Execution history data 106c is, for example, a text file. The execution history includes, for example, execution history indicating that the operation was performed normally without any processing errors, and execution history indicating that a processing error occurred.

For example, in the execution history data 106c shown in Figure 19, execution history marked with an "OK" symbol indicates that the operation was executed successfully without any processing errors, while execution history marked with an "NG" symbol indicates that a processing error occurred.

As mentioned above, the HTML file 106d is a file for visualizing the results of a test that automatically executes the developed application on the screen. The results of the test that automatically executes the application are as follows, in terms of the HTML file 106d in FIGS. 9 to 11:
(1) The "Comparison result: File name" embedded in the upper left corner of the screen ("Difference: X-XXXX-001.png" in Figure 9, "Image comparison error: X-XXXX-002.png" in Figure 10, "Runtime error: X004 X-XXXX-004" in Figure 11)
(2) The embedded image (in Figure 9, the test image and the reference image and the comparison result image, in Figure 10, the test image, and in Figure 11, no image),
Refers to...

As mentioned above, HTML file 106d is also a file that makes it easier to identify where a problem has occurred. In the case of HTML file 106d in Figures 9 to 11, the parts that make it easier to identify where a problem has occurred are the automatic execution definition information (corresponding to the contents of execution history data 106c) and a link to the definition. By looking at the automatic execution definition information, the person in charge can determine which operation caused the problem. Furthermore, by clicking the strings "UI definition information" or "execution environment," which serve as links to the definition, the person in charge can check detailed setting information for the application that was the subject of the test.

The control unit 102 is a CPU or the like that provides overall control of the test result visualization file generation device 100. The control unit 102 has internal memory for storing control programs such as an OS, programs that define various processing procedures, and required data, and executes various information processing operations based on these stored programs.

The control unit 102 has, in terms of functional concept, for example: (1) a comparison result embedding unit 102a as a comparison result embedding means for embedding, in the visualization file, the comparison result of comparing a test image, which is an image of the screen resulting from the test, with a reference image, which is a reference image to be compared, and determining whether the test image corresponds to no difference, a difference, or an incomparable state, and the file name of the test image that is determined to be different or incomparable, based on the comparison result data linked to the file name of the test image; and (2) an image embedding unit 102b as an image embedding means for embedding, in the visualization file, the test image that is determined to be different or incomparable, (3) an execution history embedding unit 102c as execution history embedding means for identifying a screen operation command including an operation command to take a screenshot of the test image determined to be different or incomparable from a screen operation command table including a screen operation command consisting of a block of operation commands that are commands for operations to be automatically executed on the screen during the test, and embedding the contents of execution history data, which is data including an execution history when an operation is executed in accordance with the identified screen operation command, into the visualization file; and (4) a processing unit 102b for confirming detailed setting information about the application that was the subject of the test based on the screen operation command identified by the execution history embedding means. (5) when an execution history indicating that a processing error has occurred exists in execution history data, which is data including an execution history when an operation is performed in accordance with the screen operation command in the screen operation command table, information for identifying the fact that a processing error has occurred, information for identifying a screen operation command including an operation command that instructed an operation corresponding to the execution history indicating that a processing error has occurred, and information for identifying a screen operation command including an operation command that instructed an operation corresponding to the execution history indicating that a processing error has occurred, are embedded in the visualization file; (5) a processing error information embedding unit 102e as processing error information embedding means for embedding in the visualization file an execution history when a processing error is detected, (6) a processing error cause embedding unit 102f as processing error cause embedding means for embedding in the visualization file an operation command that commands an operation corresponding to the execution history indicating that a processing error has occurred as the cause of the processing error, and (7) a second link embedding unit 102g as second link embedding means for embedding in the visualization file a link for checking detailed setting information for the application that was the target of the test, based on a screen operation command including an operation command that commands an operation corresponding to the execution history indicating that a processing error has occurred. The details of the processing performed by each unit are described below in [4. Specific examples of processing].

[3. Overview of Processing, etc.]
In this section, an overview of the processing in this embodiment and the terms used in this embodiment will be explained.

[3-1. Overview of processing in this embodiment]
In this embodiment, an HTML file (HTML file 106d) containing information on the comparison difference list can be generated, as shown in Fig. 3. In Fig. 3, the "Difference Image Name" enclosed in a dotted line frame indicates the definition in which the difference occurred.

Also, in Figure 3, clicking the "UI definition information" string, which is enclosed in a dotted line frame, will jump to the definition of the low-code development platform for the test case number in the difference image name. Also, in Figure 3, clicking the "Execution environment" string, which is enclosed in a dotted line frame, will jump to the definition of the execution environment for the test case number in the difference image name.

Here, in this embodiment, as a prerequisite for generating an HTML format file (HTML file 106d), the text file (comparison result data 106a) shown in Figure 4 is output as the comparison result when image comparison is performed.

In the comparison result data 106a in Figure 4, "X-XXXX-001.png," "X-XXXX-002.png," and "X-XXXX-003.png" indicate the file names of the images (test images) used for comparison. Also, in the comparison result data 106a in Figure 4, the string "Error" means that no comparison image was found (comparison was not possible) when determining whether or not there was an image to compare, the string "Difference" means that there was a difference in the image comparison, and the string "Valid" means that there was no difference in the image comparison.

In this way, in this embodiment, (1) differences between the test image and the reference image, (2) the UI definition information and execution environment for the test image, and (3) the execution history (automatic execution definition information) for analyzing the cause of the differences can be viewed in a list from an HTML file. This eliminates the need for a person in charge to manually search for information (UI definition information, execution environment, etc.) for test images that are found to have differences, a task previously performed.

[3-2. Terms used in this embodiment]
In this embodiment, when a scenario test is executed, test evidence of images and evidence of output data such as CSV data are placed in the location set in the execution setting file shown in Fig. 5. The folder structure of the evidence is, for example, as shown in Fig. 6.

The definitions of terms used in this embodiment are as follows.
・Execution log: A text file that outputs the history of scenario test execution operations.
- Image evidence: Test images for comparison taken during the scenario test.
Output data: Data that is not in image format and was output during the current scenario test. Data that is not in image format is, for example, data in CSV format.
- Baseline image evidence: A base image for comparison obtained during the previous scenario test.
Reference output data: Data that is not in image format and was output the last time a scenario test was run. Data that is not in image format is, for example, data in CSV format.
Comparison result image: An image showing the results of comparing the test image and the reference image. If there are any differences between the two, the differences are displayed, for example, highlighted in red.

4. Specific examples of processing
This section describes a specific example of processing according to this embodiment. In the following [4-1], generation of an HTML format file when an error occurs in image comparison (when comparison is not possible) is described. In the following [4-2], generation of an HTML format file when there is a difference in image comparison (when there is a difference) is described. In the following [4-3], generation of an HTML format file when there is an error in the execution log of the screen operation program (when there is a processing error) is described.

Before going into detailed processing, we will briefly outline how HTML files are generated. In this embodiment, unexpected results (image differences, runtime errors, etc.) are output as a list in an HTML file.

The generated HTML file contains the development screen (UI definition information) for the low-code development platform and a link to the low-code development platform's execution environment, allowing operators to directly check the definition content and operation from the link. In the generated HTML file, information extracted as differences is visualized by being highlighted.

Figure 9 shows an example of an HTML file generated when there are differences in the image comparison (if there are differences), Figure 10 shows an example of an HTML file generated when there is an error in the image comparison (if comparison is not possible), and Figure 11 shows an example of an HTML file generated when there is an error in the screen operation program execution log (if there is a processing error).

In this section [4. Specific examples of processing], the contents of the reference image and test image are as follows. That is, the reference image (normal image) is the image of the screen when "1" is set in From for supplier CD and "1" is automatically set in To for supplier CD. In contrast, the test image (abnormal image) is the image of the screen when "1" is set in From for supplier CD but a value is not automatically set in To for supplier CD, or when "1" is set in From for supplier CD and a value is automatically set in To for supplier CD but that value is "2."

[4-1. Creating an HTML file when an error occurs during image comparison (when comparison is not possible)]
This section explains how to generate an HTML file when an error occurs during image comparison (when comparison is not possible). An error during image comparison occurs when, for example, the image used during test execution exists but the reference image to be compared does not exist, or when the image sizes of the reference image and test image are different and comparison is not possible.

The process is explained in detail below, following the flowchart in Figure 7.

(1) Embedding a Title First, when the process begins (START in FIG. 7), the test image is compared with the reference image, and comparison result data 106a in text format is output (step SA1 in FIG. 7).

Next, based on the comparison result data 106a, the file names of images that cannot be compared or that have differences, and the status of the comparison result (Error, Difference, or Valid) are obtained (step SA2 in Figure 7).

Next, it is determined whether image comparison was successful (step SA3 in Figure 7). If image comparison was not successful (step SA3 in Figure 7: Yes), the process proceeds to step SA4, which will be described in the next paragraph. On the other hand, if image comparison was successful (step SA3 in Figure 7: No), the process proceeds to step SA9, which will be described in the next section [4-2].

If image comparison is not possible (step SA3 in FIG. 7: Yes), the comparison result embedding unit 102a embeds the comparison result indicating that comparison is not possible (the message "Image comparison error" in HTML file 106d in FIG. 12) and the file name of the test image that was determined to be incomparable (the file name "X-XXXX-002.png" in HTML file 106d in FIG. 12) into HTML file 106d based on the comparison result data 106a, as shown in FIG. 12.

In other words, the comparison result embedding unit 102a sets the file name of the image that resulted in a comparison error as the HTML title based on the image comparison result text file (step SA4 in Figure 7).

(2) Embedding of Images Next, the image embedding unit 102b embeds the test images that have been determined to be incomparable into the HTML file 106d, as shown in FIG.

In other words, the image embedding unit 102b obtains the images that resulted in a comparison error based on the image comparison result text file and embeds them into HTML (step SA5 in Figure 7).

(3) Embedding of log information and highlighting of target processing Next, the execution history embedding unit 102c identifies a screen operation command (screen operation command with definition ID = X002) from the screen operation command table 106b of Figure 14 that includes an operation command to take a screenshot of the test image that was determined to be incomparable (an operation command with type ``screenshot'', file name ``X-XXXX-002.png'', and confirmation content ``Is 1 displayed for customer CD_To?'').

Then, as shown in FIG. 14, the execution history embedding unit 102c embeds the contents of the execution history data 106c (not shown as data) when an operation is performed in accordance with the identified screen operation command into the HTML file 106d. The information indicated as "automatic execution definition information" in the HTML file 106d in FIG. 14 is the execution history data 106c after being embedded in the HTML file 106d.

In addition, as shown in FIG. 14, the execution history embedding unit 102c may also embed the confirmation content "Is 1 displayed for customer CD_To?" from the operation command (operation command with type "screenshot," file name "X-XXXX-002.png," and confirmation content "Is 1 displayed for customer CD_To?") into the HTML file 106d along with the execution history data 106c.

In this way, the execution history embedding unit 102c references the screen operation command table 106b in the business application DB based on the image comparison result text file, and embeds the confirmation details and automatic execution definition log information into HTML (step SA6 in Figure 7).

Here, as shown in FIG. 14, the execution history embedding unit 102c embeds the contents of the execution history data 106c into the HTML file 106d in a manner that makes it possible to identify the execution history "OK Take screenshot: X-XXXX-002.png" corresponding to the operation command to take a screenshot of the test image that was determined to be incomparable (shown in bold in FIG. 14).

In other words, the execution history embedding unit 102c highlights the operation command that takes a screenshot of the image that resulted in the error (step SA7 in Figure 7).

(4) Embedding of link information Finally, as shown in FIG. 15, the first link embedding unit 102d embeds links (a string of “UI definition information” and a string of “execution environment”) for checking detailed setting information about the application that was the subject of the test into the HTML file 106d based on the screen operation command (a screen operation command with definition ID=X002) identified by the execution history embedding unit 102c in (3).

In other words, the first link embedding unit 102d references the screen operation command table 106b in the business application DB based on the image comparison result text file, and embeds UI definition information and link information to the execution environment into the HTML (step SA8 in Figure 7).

Figure 16 shows an example of the screen that appears when you click on the "UI definition information" and "execution environment" strings in HTML file 106d. The operator can manually enter information based on the execution history information to check whether degradation has occurred.

[4-2. Creating an HTML file when there are differences in the image comparison (if there are differences)]
This section explains how to generate an HTML file when there are differences in the image comparison. When there are differences in the image comparison, this refers to when there are some differences between the test image and the reference image.

The process is explained in detail below, following the flowchart in Figure 7.

(1) Embedding a Title The processes from step SA1 to step SA3 are the same as those in [4-1], so detailed explanations will be omitted. If image comparison is successful in step SA3 (step SA3 in FIG. 7: No), the process proceeds to step SA9 and subsequent steps described in this section.

In step SA4, it is determined whether or not there are any differences in the image comparison (step SA9 in FIG. 7). If there are no differences (step SA9 in FIG. 7: No), processing ends (END in FIG. 7). On the other hand, if there are differences (step SA9 in FIG. 7: Yes), the comparison result embedding unit 102a embeds the comparison result showing differences (the word "Difference" in HTML file 106d in FIG. 18) and the file name of the test image determined to have a difference (the file name "X-XXXX-001.png" in HTML file 106d in FIG. 18) into HTML file 106d based on the comparison result data 106a in FIG. 17.

In other words, the comparison result embedding unit 102a sets the file name of the image that has been found to have differences as the HTML title based on the image comparison result text file (step SA10 in FIG. 7). In this way, if the images can be compared but there are differences, the comparison result embedding unit 102a obtains the file name with "Difference" added from the comparison result text file and sets the title in the HTML.

(2) Embedding of Images Next, the image embedding unit 102b embeds the test image determined to have a difference into the HTML file 106d. More specifically, as shown in FIG. 18, the image embedding unit 102b embeds the test image determined to have a difference, the reference image with which the test image was compared, and an image highlighting the difference between the two images (the "comparison result image" in the HTML file 106d in FIG. 18) into the HTML file 106d.

In other words, the image embedding unit 102b embeds the image with differences (test image), the reference image, and the comparison result image into HTML based on the image comparison result text file (step SA11 in Figure 7).

(3) Embedding of log information and highlighting of target processing Next, the execution history embedding unit 102c identifies a screen operation command (screen operation command with definition ID = X003) from the screen operation command table 106b that includes an operation command to take a screenshot of the test image that has been determined to be different (an operation command with type "screenshot", file name "X-XXXX-001.png", and confirmation content "Is 1 displayed for customer CD_To?").

Then, as shown in FIG. 18, the execution history embedding unit 102c embeds the contents of the execution history data 106c (not shown as data) when an operation is performed in accordance with the identified screen operation command into the HTML file 106d. The information indicated as "automatic execution definition information" in the HTML file 106d in FIG. 18 is the execution history data 106c after being embedded in the HTML file 106d.

In addition, as shown in FIG. 18, the execution history embedding unit 102c may also embed the confirmation content "Is 1 displayed for customer CD_To?" from the operation command (operation command with type "screenshot," file name "X-XXXX-001.png," and confirmation content "Is 1 displayed for customer CD_To?") into the HTML file 106d along with the execution history data 106c.

In this way, the execution history embedding unit 102c references the screen operation command table 106b in the business application DB based on the image comparison result text file, and embeds the confirmation details and automatic execution definition log information into HTML (step SA12 in Figure 7).

Here, as shown in FIG. 18, the execution history embedding unit 102c embeds the contents of the execution history data 106c into the HTML file 106d in a manner that makes it possible to identify the execution history "OK Take screenshot: X-XXXX-001.png" corresponding to the operation command to take a screenshot of the test image that was determined to be different (shown in bold in FIG. 18).

In other words, the execution history embedding unit 102c highlights the operation command that takes a screenshot of the image that has been found to be different (step SA13 in Figure 7).

(4) Embedding of link information Finally, as shown in FIG. 18, the first link embedding unit 102d embeds links (a string of “UI definition information” and a string of “execution environment”) for checking detailed setting information about the application that was the subject of the test into the HTML file 106d based on the screen operation command (a screen operation command with definition ID=X003) identified by the execution history embedding unit 102c in (3).

In other words, the first link embedding unit 102d references the screen operation command table 106b in the business application DB based on the image comparison result text file, and embeds UI definition information and link information to the execution environment into the HTML (step SA14 in Figure 7).

Figure 16 shows an example of the screen that appears when you click the "UI definition information" and "execution environment" strings in HTML file 106d.

The processes described in [4-1] and [4-2] above complete the generation of the HTML file 106d for a test image.

Next, a check is made to see if there are any more incomparable or different images (step SA15 in Figure 7). If there are no more incomparable or different images (step SA15 in Figure 7: No), the process ends (end in Figure 7). On the other hand, if there are still more incomparable or different images (step SA15 in Figure 7: Yes), the process returns to step SA3, and an HTML file 106d is generated for those incomparable or different images. Steps SA3 to SA15 are repeated until there are no more incomparable or different images.

[4-3. Generating an HTML file when there is an error in the execution log of the screen operation program (when there is a processing error)]
This section explains how to generate an HTML file when there is an error in the execution log of the screen operation program (when there is a processing error). Examples of when there is an error in the execution log of the screen operation program include when an error occurs during processing, when data comparison fails, and when a button cannot be pressed.

The process is explained in detail below, following the flowchart in Figure 8.

(1) Embedding a Title First, when the process starts (START in FIG. 8), the execution history data 106c (execution log) of the screen operation program is output (step SB1 in FIG. 8).

Next, information from the execution history data 106c (execution log) is obtained (step SB2 in Figure 8), and then the definition information in the execution history data 106c (execution log) is read line by line (step SB3 in Figure 8).

Next, it is determined whether a processing error (NG) exists in the execution history data 106c (execution log) of the screen operation program (step SB4 in FIG. 8). If a processing error (NG) does not exist (step SB4 in FIG. 8: No), the process ends (end in FIG. 8). On the other hand, if a processing error (NG) exists (step SB4 in FIG. 8: Yes), the process proceeds to step SB5, which will be described in the next paragraph.

As explained in the previous paragraph, if the execution history data 106c (execution log) contains an execution history indicating that a processing error occurred (step SB4 in Figure 8: Yes), the processing error information embedding unit 102e embeds information into the HTML file 106d as explained in the next paragraph based on the execution history data 106c (execution log) in Figure 19 and the screen operation command table 106b in Figure 19.

That is, the processing error information embedding unit 102e, as shown in FIG.
Information for identifying the fact that a processing error has occurred (the phrase "Runtime Error" in the HTML file 106d of FIG. 19),
Information for identifying a screen operation command (the screen operation command with definition ID = X004 in the screen operation command table 106b of Fig. 19 ) including an operation command (the operation command with type "check data", file name "X-XXXX-004_griddisplay.csv", and comparison target "X-XXXX-004_referencedata.csv" in the screen operation command table 106b of Fig. 19 ) that instructed an operation corresponding to an execution history indicating that a processing error occurred (the execution history "NG: Is the CSV data the same as the reference data?" in the execution history data 106c of Fig. 19 ), and
an execution history (automatic execution definition information in the HTML file 106d of FIG. 19) when an operation is executed in accordance with a screen operation command (screen operation command with definition ID=X004 in the screen operation command table 106b of FIG. 19) including an operation command (operation command with type “check data”, file name “X-XXXX-004_grid display.csv” and comparison target “X-XXXX-004_reference data.csv” in the screen operation command table 106b of FIG. 19) that commands an operation corresponding to an execution history indicating that a processing error has occurred (an execution history of “NG: Is the CSV data the same as the reference data?” in the execution history data 106c of FIG. 19); and
is embedded in the HTML file 106d.

In other words, the processing error information embedding unit 102e retrieves the corresponding screen operation command from the business application DB based on the failed definition name and the execution log of the screen operation program, and sets it in the HTML title. Also, the processing error information embedding unit 102e retrieves the corresponding execution history from the business application DB based on the failed definition name and the execution log of the screen operation program, and embeds it in the HTML (step SB5 in Figure 8).

Here, as shown in FIG. 19, the processing error information embedding unit 102e embeds into the HTML file 106d the execution history (automatic execution definition information in HTML file 106d in FIG. 19) when an operation is performed in accordance with a screen operation command including an operation command that commands an operation corresponding to the execution history indicating that a processing error occurred, in a manner that makes it possible to identify the execution history "NG: Is the CSV data the same as the reference data?" (shown in bold in FIG. 19).

In other words, the processing error information embedding unit 102e highlights the execution history that resulted in an NG in the execution log (step SB6 in Figure 8).

(2) Embedding of detailed information (operation command) about the processing that caused the NG The processing error cause embedding unit 102f embeds, as the cause of the processing error, an operation command (an operation command with the file name “X-XXXX-004_griddisplay.csv” in the screen operation command table 106b in FIG. 20) that commands an operation corresponding to the execution history indicating that a processing error occurred (the execution history “NG: Is the CSV data the same as the reference data?” in the execution history data 106c in FIG. 20) as shown in FIG. 20 in the HTML file 106d.

In other words, the processing error cause embedding unit 102f obtains details of the failed processing from the screen operation program log and embeds them into the HTML (step SB7 in Figure 8).

(3) Embedding of link information Finally, the second link embedding unit 102g embeds links (a string of "UI definition information" and a string of "execution environment") for checking detailed setting information about the application that was the subject of the test into the HTML file 106d based on a screen operation command (a screen operation command with definition ID = X004) that includes an operation command that commands an operation corresponding to the execution history that indicates that a processing error occurred, as shown in Figure 21.

In other words, the second link embedding unit 102g references the screen operation command table 106b in the business application DB based on the failed execution log text file, and embeds UI definition information and link information to the execution environment into the HTML (step SB8 in Figure 8).

Figure 16 shows an example of the screen that appears when you click the "UI definition information" and "execution environment" strings in HTML file 106d.

The process described in [4-3] above completes the generation of HTML file 106d for the execution history that is marked as NG in the execution history data 106c (execution log).

Next, the program returns to step SB3, where the next line is read from the definition information in the execution log. Steps SB3 to SB8 are repeated until there are no more failed execution history lines (i.e., until step SB4 proceeds to No).

5. Summary of this embodiment
As described above, the test result visualization file generation device 100 according to this embodiment can generate a visualization file (HTML file 106d) that visualizes the results of a test that automatically executes a developed application on a screen and also makes it easy to identify where a problem has occurred.

This allows the person in charge to check the test results in HTML file 106d. Furthermore, if a problem occurs during an operational test, the person in charge can check the execution history of the test operations in HTML file 106d. Furthermore, the person in charge can check detailed setting information for the developed application by clicking on a link in HTML file 106d. In other words, the person in charge can now easily and quickly both check the operational test results and identify the location of the problem simply by referring to HTML file 106d.

[6. Contribution to the United Nations-led Sustainable Development Goals (SDGs)]
This embodiment can contribute to improving business efficiency and promoting appropriate management decisions by companies, thereby contributing to the achievement of SDGs Goals 8 and 9.

Furthermore, this embodiment can contribute to reducing waste and promoting paperless and electronic systems, thereby contributing to SDGs goals 12, 13, and 15.

Furthermore, this embodiment can contribute to strengthening control and governance, thereby contributing to Goal 16 of the SDGs.

7. Other Embodiments
The present invention may be implemented in various different embodiments other than those described above within the scope of the technical concept set forth in the claims.

For example, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods.

In addition, the processing procedures, control procedures, specific names, registered data for each process, information including parameters such as search conditions, screen examples, and database configurations shown in this specification and drawings may be changed as desired unless otherwise specified.

Furthermore, with regard to the test result visualization file generation device 100, the components shown in the figure are functional concepts and do not necessarily have to be physically configured as shown.

For example, all or any part of the processing functions of the test result visualization file generation device 100, particularly the processing functions performed by the control unit, may be implemented by a CPU and a program interpreted and executed by the CPU, or may be implemented as hardware using wired logic. The program is recorded on a non-transitory, computer-readable recording medium containing programmed instructions for causing the information processing device to execute the processes described in this embodiment, and is mechanically read by the test result visualization file generation device 100 as needed. That is, a computer program that works in conjunction with the OS to issue commands to the CPU and perform various processes is recorded in a storage unit such as a ROM or HDD (Hard Disk Drive). This computer program is executed by being loaded into RAM, and works in conjunction with the CPU to form the control unit.

In addition, this computer program may be stored on an application program server connected to the test result visualization file generation device 100 via any network, and all or part of it may be downloaded as needed.

In addition, the program for executing the processing described in this embodiment may be stored on a non-temporary computer-readable recording medium, or may be configured as a program product. Here, this "recording medium" includes memory cards, USB (Universal Serial Bus) memories, SD (Secure Digital) cards, flexible disks, magneto-optical disks, ROMs, EPROMs (Erasable Programmable Read Only Memory), EEPROMs (registered trademark) (Electrically Erasable and Programmable Read Only Memory), CD-ROMs (Compact Disk Read Only Memory), MOs (Magneto-Optical disks), DVDs (Digital Versatile Disks), and more. This includes any "portable physical media" such as a Blu-ray Disc (registered trademark) and a Blu-ray (registered trademark) Disc.

A "program" is a data processing method written in any language or description method, and may be in any format, such as source code or binary code. It should be noted that a "program" is not necessarily limited to a single structure, but also includes a distributed structure consisting of multiple modules or libraries, or a structure that achieves its function by working in conjunction with a separate program, such as an OS. The specific structure and reading procedure for reading a recording medium in each device shown in the embodiments, as well as the installation procedure after reading, can use well-known structures and procedures.

The various databases stored in the memory unit are storage devices such as memory devices such as RAM and ROM, fixed disk devices such as hard disks, flexible disks, and optical disks, and store various programs, tables, databases, and web page files used for various processes and website provision.

The test result visualization file generation device 100 may be configured as an information processing device such as a known personal computer or workstation, or as an information processing device connected to any peripheral device. The test result visualization file generation device 100 may also be implemented by installing software (including programs, data, etc.) that causes the device to perform the processing described in this embodiment.

Furthermore, the specific form of distribution and integration of the devices is not limited to that shown in the figures, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various additions or functional loads. In other words, the above-mentioned embodiments may be implemented in any combination, or embodiments may be implemented selectively.

This invention is useful, for example, in the field of performing on-screen operational tests of developed applications.

100 Test result visualization file generation device 102 Control unit
102a Comparison result embedding unit
102b Image embedding unit
102c Execution history embedding unit
102d First link embedding portion
102e Processing error information embedding unit
102f Processing error cause embedding unit
102g: second link embedding unit 104: communication interface unit 106: storage unit
106a Comparison result data
106b Screen operation command table
106c Execution history data
106d HTML file 108 Input/output interface unit 112 Input device 114 Output device 200 Server 300 Network

Claims (9)

A test result visualization file generation device having a control unit that can generate a visualization file that visualizes the results of a test that automatically executes a developed application on a screen and that facilitates identification of a problem location,
The control unit
a comparison result embedding means for embedding, in the visualization file, the comparison result of whether a test image, which is an image of the screen resulting from the test, corresponds to no difference, a difference, or an incomparability, and the file name of the test image that is determined to be different or incomparable, based on comparison result data that includes the comparison result linked to the file name of the test image, when the test image, which is an image of the screen resulting from the test, is compared with a reference image, which is a reference image to be compared;
an image embedding means for embedding the test image determined to be different or incomparable into the visualization file;
an execution history embedding means for identifying a screen operation command including an operation command to take a screenshot of the test image determined to be different or incomparable from a screen operation command table including screen operation commands consisting of a block of operation commands that are commands for operations that are automatically executed on the screen during the test, and embedding the contents of execution history data, which is data including an execution history when operations are executed in accordance with the identified screen operation command, in the visualization file;
a first link embedding means for embedding a link for checking detailed setting information of the application that was the target of the test into the visualization file based on the screen operation command identified by the execution history embedding means;
To have
A test result visualization file generation device characterized by the above. The execution history embedding means
Embedding the content of the execution history data into the visualization file in a manner that allows identification of an execution history corresponding to an operation command instructing to take a screenshot of the test image that is determined to be different or incomparable;
2. The test result visualization file generating device according to claim 1, wherein: The image embedding means
Embedding the test image determined to have a difference, the reference image used to compare the test image, and an image highlighting the difference between the two images into the visualization file;
2. The test result visualization file generating device according to claim 1, wherein: The control unit
a processing error information embedding means for, when an execution history indicating that a processing error has occurred exists in execution history data, which is data including an execution history when an operation is performed in accordance with the screen operation command in the screen operation command table, embedding, into the visualization file, information for identifying the fact that a processing error has occurred, information for identifying a screen operation command including an operation command which instructed an operation corresponding to the execution history indicating that a processing error has occurred, and an execution history when an operation is performed in accordance with a screen operation command including an operation command which instructed an operation corresponding to the execution history indicating that a processing error has occurred, based on the execution history data and the screen operation command table;
a processing error cause embedding means for embedding, into the visualization file, an operation command that commands an operation corresponding to an execution history indicating that a processing error has occurred, as the cause of the processing error;
a second link embedding means for embedding a link in the visualization file for checking detailed setting information of the application that was the target of the test, based on a screen operation command including an operation command for instructing an operation corresponding to an execution history indicating that a processing error has occurred;
Further comprising:
2. The test result visualization file generating device according to claim 1, wherein: The processing error information embedding means
Embedding, in the visualization file, an execution history when an operation is performed in accordance with a screen operation command including an operation command that commands an operation corresponding to the execution history indicating that a processing error has occurred, in a manner that enables identification of the execution history indicating that a processing error has occurred;
5. The test result visualization file generating device according to claim 4, wherein: The visualization file is a text file;
6. The test result visualization file generating device according to claim 1, wherein: The text file is an HTML file.
7. The test result visualization file generating device according to claim 6, wherein: A test result visualization file generation method executed by an information processing device having a control unit, which is capable of generating a visualization file that visualizes the results of a test in which a developed application is automatically executed on a screen and that facilitates identification of a problem location,
Executed by the control unit,
a comparison result embedding step of embedding the comparison result of a test image, which is an image of the screen resulting from the test, into the visualization file the file name of the test image that is determined to be different or incomparable, based on comparison result data that includes the comparison result, which is the result of comparing a test image, which is an image of the screen resulting from the test, with a reference image, which is a reference image to be compared, and determining whether the test image corresponds to no difference, a difference, or incomparable, and the file name of the test image that is determined to be different or incomparable;
an image embedding step of embedding the test image determined to be different or incomparable into the visualization file;
an execution history embedding step of identifying a screen operation command including an operation command to take a screenshot of the test image determined to be different or incomparable from a screen operation command table including screen operation commands consisting of a block of operation commands that are commands for operations that are automatically executed on the screen during the test, and embedding the contents of execution history data, which is data including an execution history when operations are executed in accordance with the identified screen operation command, into the visualization file;
a first link embedding step of embedding a link for checking detailed setting information of the application that was the target of the test into the visualized file based on the screen operation command identified in the execution history embedding step;
containing,
A test result visualization file generation method characterized by the above. A test result visualization file generation program for an information processing device having a control unit, which is capable of generating a visualization file that visualizes the results of a test that automatically executes a developed application on a screen and also facilitates identification of a problem location,
To cause the control unit to execute
a comparison result embedding step of embedding the comparison result of a test image, which is an image of the screen resulting from the test, into the visualization file the file name of the test image that is determined to be different or incomparable, based on comparison result data that includes the comparison result, which is the result of comparing a test image, which is an image of the screen resulting from the test, with a reference image, which is a reference image to be compared, and determining whether the test image corresponds to no difference, a difference, or incomparable, and the file name of the test image that is determined to be different or incomparable;
an image embedding step of embedding the test image determined to be different or incomparable into the visualization file;
an execution history embedding step of identifying a screen operation command including an operation command to take a screenshot of the test image determined to be different or incomparable from a screen operation command table including screen operation commands consisting of a block of operation commands that are commands for operations that are automatically executed on the screen during the test, and embedding the contents of execution history data, which is data including an execution history when operations are executed in accordance with the identified screen operation command, into the visualization file;
a first link embedding step of embedding a link for checking detailed setting information of the application that was the target of the test into the visualized file based on the screen operation command identified in the execution history embedding step;
containing,
A test result visualization file generator featuring: