JP7544142B2 - OFFLOAD SERVER, OFFLOAD CONTROL METHOD, AND OFFLOAD PROGRAM - Google Patents

Description

The present invention relates to an offload server, an offload control method, and an offload program that automatically offloads functional processing to an accelerator such as an FPGA (Field Programmable Gate Array).

In recent years, it has been said that Moore's Law, which states that the semiconductor integration density of CPUs will double every 1.5 years, may be slowing down. In light of this situation, there has been an increase in the use of devices such as FPGAs (Field Programmable Gate Arrays) and GPUs (Graphics Processing Units) in addition to CPUs with fewer cores. For example, Microsoft (registered trademark) is working to improve the search efficiency of Bing using FPGAs, and Amazon (registered trademark) is providing FPGAs, GPUs, etc. as cloud instances.

In order to properly utilize devices other than low-core CPUs in a system, it is necessary to configure and write programs that take into account the characteristics of the device, and knowledge of OpenMP (Open Multi-Processing), OpenCL (Open Computing Language), and CUDA (Compute Unified Device Architecture) is required, which creates a high skill barrier for most programmers.

It is expected that the number of systems that utilize devices other than few-core CPUs, such as GPUs, FPGAs, and many-core CPUs, will continue to increase in the future, but there are high technical barriers to making the most of them. Therefore, in order to remove such barriers and make full use of devices other than few-core CPUs, there is a demand for a platform that can adaptively convert and configure software in which a programmer writes processing logic to suit the environment in which it is deployed (FPGA, GPU, many-core CPU, etc.), allowing it to operate in accordance with the environment.

Non-patent document 1 describes environment-adaptive software that automatically converts and configures resources so that code written once can utilize GPUs, FPGAs, many-core CPUs, etc. present in the deployment environment, with the aim of making applications run with high performance.

Non-patent documents 2, 3, and 4 describe a method for automatically offloading loop statements and function blocks of application code to FPGAs and GPUs as elements of environment-adaptive software. Non-patent document 5 describes a technology for automatic performance testing.

CUDA has become popular as an environment for running GPGPUs (General Purpose GPUs), which use the parallel computing power of GPUs for purposes other than image processing. CUDA is an environment for GPGPUs developed by NVIDIA (registered trademark). In addition, there is OpenCL, a specification for treating heterogeneous devices such as FPGAs, many-core CPUs, and GPUs in the same way, and its development environment is also in the process of being established. CUDA and OpenCL are programmed by extending the C language, and are difficult to program (such as explicitly writing the copying and release of memory data between the kernel of an FPGA or the like and the host CPU).

In addition, compared to CUDA and OpenCL, there are technologies such as OpenACC and OpenMP (registered trademark) that allow easier use of heterogeneous devices, and compilers such as the PGI compiler and gcc (registered trademark). These compilers specify the locations where parallel processing will be performed on a directive line basis, and create executable files for GPUs, many-core CPUs, etc. according to the directive lines.

By using the above technical specifications such as CUDA, OpenCL, OpenACC, and OpenMP, it is possible to offload to FPGAs, GPUs, and many-core CPUs. However, even if device processing itself can be performed, there are challenges in accelerating it. For example, Intel Compiler (registered trademark) is a compiler with an automatic parallelization function for multi-core CPUs. During automatic parallelization, these extract parts of the loop statements in the code that can be parallelized and parallelize them. However, due to the influence of memory processing, etc., simply parallelizing parallelizable loop statements often does not achieve sufficient performance. When accelerating FPGAs, GPUs, etc., OpenCL and CUDA engineers repeatedly tune the system, or use OpenACC compilers, etc. to search for and try out the appropriate range of parallel processing.

For this reason, it is difficult for programmers with little technical skill to use FPGAs, GPUs, or many-core CPUs to speed up applications, and even when using automatic parallelization technology, it is necessary to go through a process of trial and error to find parallel processing locations. Currently, offloading to heterogeneous devices is mainly done manually.

Y. Yamato, H. Noguchi, M. Kataoka and T. Isoda, “Proposal of Environment Adaptive Software,” The 2nd International Conference on Control and Computer Vision (ICCCV 2019), pp.102-108, Jeju, June 2019. Y. Yamato, “Study of parallel processing area extraction and data transfer number reduction for automatic GPU offloading of IoT applications,” Journal of Intelligent Information Systems, Springer, DOI: 10.1007/s10844-019-00575-8, Aug. 2019. Y. Yamato, “Proposal of Automatic FPGA Offloading for Applications Loop Statements,” The 7th Annual Conference on Engineering and Information Technology (ACEAIT 2020), pp.111-123, 2020. Y. Yamato, “Proposal of Automatic Offloading for Function Blocks of Applications,” The 8th IIAE International Conference on Industrial Application Engineering 2020 (ICIAE 2020), pp.4-11, Mar. 2020. Y. Yamato, "Automatic verification technology of software patches for user virtual environments on IaaS cloud," Journal of Cloud Computing, Springer, 2015, 4:4, DOI: 10.1206/s13677-015-0028-6, Feb. 2015.

When a program for a CPU is offloaded to a device such as a GPU, in order to operate the application with high cost performance, it is required to optimize the resource amounts of the CPU and the offload device.
Non-Patent Documents 1 to 5 do not consider the amount of resources of the offload device after automatic offloading.

The present invention was made in consideration of these points, and its objective is to enable appropriate resource amounts to be set for the CPU and offload device in automatic offloading.

In order to solve the above-mentioned problems, an offload server that offloads specific processing of an application program from a CPU to an offload device includes an application code analysis unit that analyzes source code of the application program, a data transfer specification unit that analyzes reference relationships of variables used in loop statements of the application program, and for data that may be transferred outside the loop, specifies a data transfer using an explicit specification line that explicitly specifies data transfer outside the loop, a parallel processing specification unit that identifies loop statements of the application program and compiles each identified loop statement by specifying a parallel processing specification statement in the offload device, and a parallel processing pattern that excludes loop statements that generate a compilation error from being offloaded and specifies whether or not to process loop statements that do not generate a compilation error in parallel. the offload server includes a parallel processing pattern creation unit that creates a parallel processing pattern based on the parallel processing pattern; a performance measurement unit that compiles the application program of the parallel processing pattern, places it on an accelerator verification device, and executes processing for measuring performance when offloaded to the offload device; a resource ratio determination unit that determines a ratio of processing times of the CPU and the offload device as a resource ratio based on the performance measurement results; and a resource amount setting unit that sets the resource amounts of the CPU and the offload device so as to satisfy a specified cost condition based on the determined resource ratio, wherein if the specified cost condition is not satisfied by setting a minimum resource amount that maintains the determined resource ratio, the resource amount setting unit sets the resource amounts of the CPU and the offload device so as to approach the set resource ratio starting from a minimum while satisfying the cost condition .

According to the present invention, appropriate resource amounts for the CPU and offload device can be set during automatic offloading.

FIG. 2 is a functional block diagram illustrating a configuration example of an offload server according to the first embodiment of the present invention. FIG. 2 is a diagram illustrating an automatic offload process using an offload server according to the first embodiment of the present invention. 1 is a diagram showing a search image of processing by a control unit (automatic offload function unit) by Simple GA of an offload server according to a first embodiment of the present invention, and gene sequence mapping of a for statement; FIG. 1 is a flowchart of a loop statement offload of an offload server according to the first embodiment of the present invention. 1 is a flowchart of a loop statement offload of an offload server according to the first embodiment of the present invention. 11 is a flowchart for setting a resource ratio and a resource amount added after a GPU or FPGA offload attempt of an offload server according to the first embodiment of the present invention. FIG. 11 is a functional block diagram illustrating a configuration example of an offload server according to a second embodiment of the present invention. This is a flowchart when the control unit (automatic offload function unit) of the offload server related to the second embodiment of the present invention executes <Process A-1>, <Process B-1>, and <Process C-1> in the offload processing of <Function block offload: common>. This is a flowchart when a control unit (automatic offload function unit) of an offload server relating to the second embodiment of the present invention executes <Process A-2>, <Process B-2>, and <Process C-2> in the offload processing of a functional block. FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of an offload server according to an embodiment of the present invention.

Next, an offload server and the like in an embodiment of the present invention (hereinafter, referred to as "the present embodiment") will be described.
(Basic concept of the present invention)
In order to materialize the concept of environment adaptive software, the present inventor has proposed automatic offloading of program loop statements to GPUs, automatic offloading of FPGAs, and automatic offloading of program function blocks (see Non-Patent Documents 2, 3, and 4). Taking into account the elemental technology studies in these Non-Patent Documents 2, 3, and 4, the basic concept of the present invention will be described.

<Optimizing the resource ratio between CPU and offload device>
First, an explanation will be given of optimizing the resource ratio between the CPU and the offload device after a program has been converted for offloading to a device.
According to the technique disclosed in Non-Patent Document 2, it is possible to automatically offload normal programs to an offload device such as a GPU or an FPGA.
Currently, multi-core and many-core CPUs are virtualized using virtual machines and containers, allowing for flexibility in allocating a certain percentage of all cores. GPUs have also been virtualized in recent years in a similar manner to CPUs, making it possible to allocate a certain percentage of all GPU cores. With regard to FPGAs, resource usage is often expressed as the number of look-up tables or flip-flops set, and unused gates can be used for other purposes.

In this way, it is possible to operate the CPU, GPU, and FPGA using only a portion of their total resources, and optimizing the resources of the CPU and offload devices according to the application is important in improving cost performance.

It is also possible to convert an application into code for CPU and GPU processing using techniques such as those described in Non-Patent Document 2. However, even if the code itself is appropriate, performance will not be achieved if the resource amounts of the CPU and GPU are not properly balanced. For example, when performing a certain process, if the CPU processing time is 1000 seconds and the GPU processing time is 1 second, even if the GPU speeds up the process that can be offloaded to some extent, the CPU will still be the bottleneck overall.

Furthermore, in the non-patent document "K. Shirahata, H. Sato and S. Matsuoka, "Hybrid Map Task Scheduling for GPU-Based Heterogeneous Clusters," IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), pp.733-740, Dec. 2010," it is described how, when processing tasks using a CPU and GPU in the MapReduce (registered trademark) framework, map tasks are allocated so that the execution times of the CPU and GPU are the same, thereby improving overall performance.

The inventors came up with the idea of determining the resource ratio between the CPU and the offload device as follows: In other words, to avoid processing in any of the devices becoming a bottleneck, and with reference to the above non-patent literature, the resource ratio between the CPU and the offload device (hereinafter referred to as "resource ratio") is determined based on the processing time of the test case so that the processing time of the CPU and the offload device are of the same order of magnitude.

The inventor also employs a method of gradually increasing the speed of automatic offloading based on the results of performance measurements in a verification environment, as in the method in Non-Patent Document 2. The reason for this is that performance varies greatly not only depending on the code structure, but also on the specifications of the hardware used for the actual processing, data size, loop count, and other content actually processed. Also, performance is difficult to predict statically, and dynamic measurement is necessary. Therefore, since performance measurement results are already available in the verification environment when converting code, the resource ratio is determined using those results.

When measuring performance, a test case is specified and the measurement is performed. For example, if the processing time of a test case in the verification environment is 10 seconds for CPU processing and 5 seconds for GPU processing, then the CPU resources would be doubled to achieve approximately the same processing time. Therefore, the resource ratio is 2:1. Note that if a user wishes to speed up a particular process by offloading, this can be reflected by preparing a test case that includes that process and speeding up the test case using a method such as that described in Non-Patent Document 2.

<Determining and automatically validating the amount of CPU and offload device resources>
Next, determination and automatic verification of the resource amounts of the CPU and offload devices (hereinafter referred to as "resource amounts") will be described.
When the resource ratio has been determined by the above <Optimizing the resource ratio between the CPU and the offload device>, the next step is to deploy the application in the commercial environment.
When placing in a commercial environment, the resource amount is determined while keeping (maintaining) the resource ratio as much as possible so as to satisfy the cost requirement specified by the user. For example, for the CPU, it is assumed that 1 VM costs 1000 yen/month, GPU costs 4000 yen/month, and the resource ratio is 2:1. It is also assumed that the user's budget is within 10,000 yen/month. In this case, even if the resource ratio is 2:1, it is within the user's budget of 10,000 yen/month, so the resource amount that keeps the appropriate resource ratio of 2:1, that is, "2" for the CPU and "1" for the GPU, is secured and placed in the commercial environment. Also, if the user's budget is within 5,000 yen/month, the appropriate resource ratio of 2:1 cannot be kept. In this case, the resource amount is secured and placed for the CPU and GPU, with "1" secured.

After reserving resources in the commercial environment and placing the program, automatic verification is performed to ensure that it works before users use it. In automatic verification, performance verification test cases and regression test cases are executed. For performance verification test cases, assumed test cases specified by the user are executed using an automatic test execution tool such as Jenkins (registered trademark), and processing time, throughput, etc. are measured. For regression test cases, information on software such as middleware and the OS installed in the system is obtained, and the corresponding regression tests are executed using Jenkins or the like. Studies on how to perform these automatic verifications with the preparation of fewer test cases have been conducted in non-patent document 5 and other documents, and the technology in non-patent document 5 is used.

The performance verification test cases check whether the calculation results are invalid even when offloading is performed. The performance verification test cases also check the difference in the calculation results from when offloading is not performed. For example, the PGI compiler that processes the GPU can check the difference in the calculation results when the GPU is used and when it is not, using APIs (Application Programming Interfaces) called PGI_compare (registered trademark) and acc_compare (registered trademark) of the PCAST (registered trademark) function.
Note that there are cases where the calculation results do not match completely even if parallel processing is offloaded correctly due to differences in rounding errors between the GPU and the CPU. For this reason, for example, confirmation based on the IEEE 754 specifications is performed, and the user is informed of the allowable difference and asked to confirm it.

As a result of the automated verification, the user is presented with information such as the processing time and throughput of the performance verification test cases, the difference in the calculation results, and the results of the regression tests. The user is also presented with the secured resources (number of VMs, their specifications, etc.) and their prices, and can refer to this information to decide whether to start operation.

<Resources, resource ratios, test case processing time>
The resources, resource ratios, and test case processing times in this embodiment will be described.
Regarding resources, CPUs, GPUs, FPGAs, etc. are now provided as instances of virtual resources.
Resources include the number of CPU cores, clock, memory amount, disk size, the number of GPU cores, clock, memory amount, and FPGA gate size (LE (registered trademark) for Intel (registered trademark), LC (registered trademark) for Xilinx (registered trademark)). Cloud service providers package these resources and provide them in the form of small-sized virtual machines and GPU instances. When virtualizing, the number of instances used can be said to be the amount of resources used.

Regarding resource ratio, the ratio of the number of instances of the CPU, GPU, and FPGA is the resource ratio. If the number of instances is 1, 2, or 3, the resource ratio is 1:2:3.

- Test case processing time In this embodiment, an offload pattern that speeds up a test case specified by a user is searched for and found. A test case is a transaction processing number such as TPC-C (registered trademark) for DB (database), and a Fourier transform processing execution on sample data for FFT. The processing time is the execution time when the sample processing is executed. For example, the processing time of processing A was 10 seconds before offloading, but became 2 seconds after offloading, and the execution times when it was executed by the CPU and when it was executed by the offload device are respectively obtained.

First Embodiment
The present invention is characterized by the setting of resource ratios and resource amounts in automatic offloading.
In the first embodiment, the setting of resource ratios and resource amounts in offloading a loop statement will be described. First, the offloading of a loop statement will be described.
FIG. 1 is a functional block diagram showing an example of the configuration of an offload server 1 according to the first embodiment of the present invention.
The offload server 1 is a device that automatically offloads specific processing of an application to an accelerator.
As shown in FIG. 1, the offload server 1 includes a control unit 11, an input/output unit 12, a storage unit 13, and a verification machine 14 (accelerator verification device).

The input/output unit 12 is composed of a communication interface for transmitting and receiving information between each device, etc., and an input/output interface for transmitting and receiving information between input devices such as a touch panel or keyboard, and output devices such as a monitor.

The memory unit 13 is composed of a hard disk, flash memory, RAM (Random Access Memory), etc., and temporarily stores programs (offload programs) for executing each function of the control unit 11, and information necessary for processing by the control unit 11 (e.g., intermediate language file 133).

The memory unit 13 includes a test case database 131, an equipment resource DB 132, and an intermediate language file 133.

Test case DB131 stores data on test items corresponding to the software to be verified. For example, in the case of a database system such as MySQL, the test item data is transaction test data such as TPC-C.

The equipment resource DB132 holds information prepared in advance about resources such as servers held by the business operator, such as their prices, and information about how much they are being used. For example, there are 10 servers that can accommodate three GPU instances, one GPU instance costs 5,000 yen per month, and of the 10, three servers A, B, and C are being used. This information is used to determine the amount of resources to be secured when the user specifies operational conditions (cost, performance, etc.). The user operational conditions are the cost conditions (e.g., a monthly budget of 10,000 yen or less) and performance conditions (e.g., a transaction throughput of TPC-C or the like, or a sample Fourier transform processing time of one thread or less, etc.) specified by the user when requesting offloading.

The intermediate language file 133 temporarily stores information necessary for processing by the control unit 11 in the form of a programming language that is intermediate between a high-level language and a machine language.

The verification machine 14 is equipped with a CPU, GPU, and FPGA as a verification environment for the environmentally adaptive software.

The control unit 11 is an automatic offloading function that controls the entire offload server 1. The control unit 11 is realized, for example, by a central processing unit (CPU) (not shown) that deploys an application program (offload program) stored in the memory unit 13 into RAM and executes it.

The control unit 11 includes an application code specification unit 111, an application code analysis unit 112, a data transfer specification unit 113, a parallel processing specification unit 114, a resource ratio determination unit 115, a resource amount setting unit 116, a parallel processing pattern creation unit 117, a performance measurement unit 118, an executable file creation unit 119, a production environment deployment unit 120, a performance measurement test extraction and execution unit 121, and a user provision unit 122.

<Application code specification unit 111>
The application code designation unit 111 designates the input application code. Specifically, the application code designation unit 111 passes the application code described in the received file to the application code analysis unit 112.

<Application Code Analysis Unit 112>
The application code analysis unit 112 analyzes the source code of the processing function and understands the structure of loop statements, FFT library calls, and the like.

<Data transfer designation unit 113>
The data transfer specification unit 113 analyzes the reference relationships of variables used in loop statements of an application program, and for data that may be transferred outside the loop, specifies the data transfer using an explicit specification line that explicitly specifies the data transfer outside the loop.

<Parallel processing designation unit 114>
The parallel processing specification unit 114 identifies loop statements (repetitive statements) in the application program, and for each loop statement, specifies a parallel processing specification statement for the accelerator and compiles it.
The parallel processing specification unit 114 includes an offloadable area extraction unit (Extract offloadable area) 114a and an intermediate language file output unit (Output intermediate file) 114b.

The offload range extraction unit 114a identifies processes that can be offloaded to the GPU/FPGA, such as loop statements and FFTs, and extracts an intermediate language corresponding to the offloaded process.
The intermediate language file output unit 114b outputs the extracted intermediate language file 133. The intermediate language extraction is not a one-time process, but is repeated to perform trial and error optimization to search for an appropriate offload area.

<Resource Ratio Determination Unit 115>
The resource ratio determination unit 115 determines the processing times of the CPU and the offload device (test case CPU processing time and offload device processing time) as a resource ratio based on the performance measurement results (described later). Specifically, the resource ratio determination unit 115 determines the resource ratio so that the processing times of the CPU and the offload device are of the same order. Furthermore, when the difference between the processing times of the CPU and the offload device is equal to or greater than a predetermined threshold, the resource ratio determination unit 115 sets the resource ratio to a predetermined upper limit value.

<Resource Amount Setting Unit 116>
The resource amount setting unit 116 sets the resource amounts of the CPU and offload device based on the determined resource ratio so as to satisfy a predetermined cost condition (described later). Specifically, the resource amount setting unit 116 maintains the determined resource ratio and sets the maximum resource amount that satisfies the predetermined cost condition. Furthermore, if the resource amount setting unit 116 does not satisfy the predetermined cost condition by setting the minimum resource amount while maintaining the determined resource ratio, it breaks the resource ratio and sets the resource amounts of the CPU and offload device to a value smaller (e.g., minimum) that satisfies the cost condition.

<Parallel processing pattern creation unit 117>
The parallel processing pattern creation unit 117 creates a parallel processing pattern that excludes loop statements (repetitive statements) that cause a compilation error from being offloaded, and specifies whether or not to process repetitive statements that do not cause a compilation error in parallel.

<Performance Measuring Unit 118>
The performance measurement unit 118 compiles an application program of a parallel processing pattern, places it on the verification machine 14, and executes a process for measuring performance when offloaded to the accelerator.
The performance measurement unit 118 includes a binary file deployment unit 118a. The binary file deployment unit 118a deploys an executable file derived from the intermediate language on the verification machine 14 including a GPU and FPGA.

The performance measurement unit 118 executes the placed binary file, measures the performance when offloaded, and returns the performance measurement results to the offload range extraction unit 114a. In this case, the offload range extraction unit 114a extracts another parallel processing pattern, and the intermediate language file output unit 114b attempts to measure performance based on the extracted intermediate language (see symbol a in Figure 2 below).

<Executable File Creation Unit 119>
The executable file creation unit 119 selects multiple parallel processing patterns with high processing performance from the multiple parallel processing patterns based on the performance measurement results repeated a predetermined number of times, crosses the parallel processing patterns with high processing performance, and creates multiple other parallel processing patterns by mutation processing.The executable file creation unit 119 then performs new performance measurements, and after the performance measurements have been performed a specified number of times, selects the parallel processing pattern with the highest processing performance from the multiple parallel processing patterns based on the performance measurement results, compiles the parallel processing pattern with the highest processing performance, and creates an executable file.

<Production Environment Deployment Unit 120>
The production environment deployment unit 120 deploys the created executable file in the production environment for the user ("Deployment of final binary file in production environment"). The production environment deployment unit 120 determines a pattern that specifies the final offload area, and deploys it to the production environment for the user.

<Performance Measurement Test Extraction Execution Unit 121>
After placing the executable file, the performance measurement test extraction execution unit 121 extracts performance test items from the test case DB 131 and executes the performance tests ("Placing the final binary file in the production environment").
After arranging the executable file, the performance measurement test extraction execution unit 121 extracts performance test items from the test case DB 131 and automatically executes the extracted performance tests in order to show the performance to the user.

<User provision unit 122>
The user providing unit 122 presents information such as price and performance based on the performance test results to the user ("Providing information such as price and performance to the user"). Performance test items are stored in the test case DB 131. The user providing unit 122 presents data such as price and performance to the user along with the performance test results based on the results of the performance tests corresponding to the test items stored in the test case DB 131. The user decides whether to start charging for the service based on the presented information such as price and performance. Here, the technology in non-patent document (Y. Yamato, M. Muroi, K. Tanaka and M. Uchimura, “Development of Template Management Technology for Easy Deployment of Virtual Resources on OpenStack,” Journal of Cloud Computing, Springer, 2014, 3:7, DOI: 10.1206/s13677-014-0007-3, 12 pages, June 2014.) can be used for bulk deployment to the production environment, and the technology in non-patent document 5 can be used for automatic performance testing.

[Application of genetic algorithm]
The offload server 1 can use GA (Genetic Algorithms) for optimizing the offload. The offload server 1 when using GA is configured as follows.
That is, the parallel processing specification unit 114 sets the number of loop statements (repetitive statements) that do not cause a compilation error as the gene length based on a genetic algorithm. The parallel processing pattern creation unit 117 maps the availability of accelerator processing to the gene pattern by setting either 1 or 0 when accelerator processing is performed and the other 0 or 1 when accelerator processing is not performed.

The parallel processing pattern creation unit 117 prepares a specified number of gene patterns in which each gene value is randomly created to be 1 or 0. The performance measurement unit 118 compiles application code that specifies a parallel processing specification statement in the accelerator according to each individual, and places it on the verification machine 14. The performance measurement unit 118 executes a performance measurement process on the verification machine 14.

Here, if a gene with the same parallel processing pattern as before occurs in an intermediate generation, the performance measurement unit 118 does not compile the application code corresponding to that parallel processing pattern or measure its performance, but uses the same value as the performance measurement value.
Furthermore, for application code that causes a compilation error and application code for which performance measurement does not end within a predetermined time, the performance measurement unit 118 treats the application code as a timeout and sets the performance measurement value to a predetermined time (long time).

The executable file creation unit 119 measures the performance of all the individuals, and evaluates them so that the shorter the processing time, the higher the fitness. From all the individuals, the executable file creation unit 119 selects those with a fitness higher than a predetermined value (e.g., the top n% of the total number, or the top m of the total number, where n and m are natural numbers) as high-performance individuals, and performs crossover and mutation processes on the selected individuals to create the next generation of individuals. After completing processing for the specified number of generations, the executable file creation unit 119 selects the parallel processing pattern with the highest performance as the solution.

The automatic offload operation of the offload server 1 configured as above will now be described.
[Automatic offload operation]
FIG. 2 is a diagram showing an automatic offload process using the offload server 1. As shown in FIG.
2, the offload server 1 is applied to the elemental technology of the environment adaptive software. The offload server 1 has a control unit (automatic offload function unit) 11, a test case DB 131, a facility resource DB 132, an intermediate language file 133, and a verification machine 14.
The offload server 1 acquires an application code 125 used by the user.

The user is, for example, a person who has signed a contract to use various devices (Device 151, device having CPU-GPU 152, device having CPU-FPGA 153, device having CPU 154).
The offload server 1 automatically offloads function processing to accelerators such as a device 152 having a CPU-GPU and a device 153 having a CPU-FPGA.

The operation of each unit will be described below with reference to the step numbers in FIG.
<Step S11: Specify application code>
In step S 11 , the application code designation unit 111 (see FIG. 1 ) passes the application code described in the received file to the application code analysis unit 112 .

<Step S12: Analyze application code>
In step S12, the application code analysis unit 112 (see FIG. 1) analyzes the source code of the processing function and grasps the structures of loop statements, FFT library calls, and the like.

<Step S13: Extract offloadable area>
In step S13, the parallel processing specification unit 114 (see FIG. 1) identifies loop statements (repetitive statements) in the application, and compiles each repetitive statement by specifying a parallel processing specification statement in the accelerator. Specifically, the offload range extraction unit 114a (see FIG. 1) identifies processes that can be offloaded to the GPU/FPGA, such as loop statements and FFT, and extracts an intermediate language corresponding to the offloaded process.

<Step S14: Output intermediate file>
In step S14, the intermediate language file output unit 114b (see FIG. 1) outputs the intermediate language file 133. The intermediate language extraction is not a one-time process, but is repeated to try and optimize the execution for an appropriate offload area search.

<Step S15: Compile error>
In step S15, the parallel processing pattern creation unit 117 (see FIG. 1 ) creates a parallel processing pattern that excludes loop statements that produce a compilation error from being offloaded, and specifies whether or not to perform parallel processing on repetitive statements that do not produce a compilation error.

<Step S21: Deploy binary files>
In step S21, the binary file placement unit 118a (see FIG. 1) deploys an executable file derived from the intermediate language to the verification machine 14 equipped with a GPU/FPGA.

<Step S22: Measure performance>
In step S22, the performance measurement unit 118 (see FIG. 1) executes the arranged file and measures the performance when offloaded.
In order to more appropriately determine the area to be offloaded, the performance measurement result is returned to the offload range extraction unit 114a, which extracts another pattern. The intermediate language file output unit 114b then attempts to measure performance based on the extracted intermediate language (see symbol a in FIG. 2).

As shown by the symbol a in FIG. 2, the control unit 11 repeatedly executes steps S12 to S22. The automatic offload function of the control unit 11 can be summarized as follows. That is, the parallel processing designation unit 114 identifies loop statements (repeated statements) of the application program, and compiles each repeated statement by designating a parallel processing designation statement in the GPU. The parallel processing pattern creation unit 117 creates a parallel processing pattern that excludes loop statements that generate compilation errors from offloading targets and designates whether or not to perform parallel processing for loop statements that do not generate compilation errors. The binary file placement unit 118a then compiles the application program of the corresponding parallel processing pattern and places it on the verification machine 14, and the performance measurement unit 118 executes the performance measurement process on the verification machine 14. The executable file creation unit 119 selects a pattern with the highest processing performance from a plurality of parallel processing patterns based on the performance measurement results repeated a predetermined number of times, and compiles the selected pattern to create an executable file.

<Step S23: Setting resource amount according to user operation conditions>
In step S23, the control unit 11 sets the resource amount according to the user operation conditions. That is, the resource ratio determination unit 115 of the control unit 11 determines the resource ratio between the CPU and the offload device. Then, the resource amount setting unit 116 refers to the information in the facility resource DB 132 based on the determined resource ratio, and sets the resource amount of the CPU and the offload device so as to satisfy the user operation conditions (described later with reference to FIG. 5).

<Step S24: Deploy final binary files to production environment>
In step S24, the production environment deployment unit 120 determines a pattern that specifies the final offload area, and deploys it to the production environment for the user.

<Step S25: Extract performance test cases and run automatically>
In step S25, after arranging the executable file, the performance measurement test extraction execution unit 121 extracts performance test items from the test case DB 131 and automatically executes the extracted performance tests in order to show the performance to the user.

<Step S26: Provide price and performance to a user to judge>
In step S26, the user providing unit 122 provides the user with information on price, performance, etc., based on the performance test results. The user determines whether to start using the service for which charges are required, based on the provided information on price, performance, etc.

The above steps S11 to S26 are assumed to be carried out, for example, in the background while the user is using the service, for example during the first day of trial use.

As described above, when applied to the elemental technology of the environment-adaptive software, the control unit (automatic offload function unit) 11 of the offload server 1 extracts the area to be offloaded from the source code of the application program used by the user and outputs an intermediate language to offload functional processing (steps S11 to S15). The control unit 11 places and executes the executable file derived from the intermediate language on the verification machine 14 and verifies the offloading effect (steps S21 to S22). After repeating the verification and determining an appropriate offload area, the control unit 11 deploys the executable file in the production environment that will actually be provided to the user and provides it as a service (steps S23 to S26).

[Automatic GPU offloading using GA]
The GPU automatic offload is a process in which steps S12 to S22 in FIG. 2 are repeated for the GPU to obtain the offload code to be deployed in step S23.

GPUs generally do not guarantee latency, but are devices suited to increasing throughput through parallel processing. Typical applications include encryption processing, image processing for analyzing camera footage, and machine learning processing for analyzing large amounts of sensor data, which involve a lot of repetitive processing. Therefore, the aim is to increase speed by automatically offloading repetitive statements in applications to the GPU.

However, as described in the prior art, appropriate parallel processing is necessary to increase speed. In particular, when using a GPU, due to memory transfer between the CPU and GPU, performance is often not achieved unless the data size and number of loops are large. Also, depending on the timing of memory data transfer, the combination of individual loop statements (repeated statements) that can be accelerated in parallel may not be the fastest. For example, if there are 10 "for" statements (repeated statements), and numbers 1, 5, and 10 can be accelerated compared to the CPU, the combination of numbers 1, 5, and 10 will not necessarily be the fastest.

In order to specify appropriate parallel regions, there have been attempts to optimize the parallelism of for statements through trial and error using the PGI compiler. However, trial and error requires a lot of work, and when offered as a service, this can delay users' start-up and increase costs.

In this embodiment, therefore, suitable offload areas are automatically extracted from general-purpose programs that do not assume parallelization. For this purpose, parallelizable for statements are checked first, and then performance verification trials are repeated in a verification environment using GA for the group of parallelizable for statements to search for suitable areas. By narrowing down to parallelizable for statements and then retaining and recombining parallel processing patterns that can be accelerated in the form of genetic parts, it is possible to efficiently search for patterns that can be accelerated from the enormous number of possible parallel processing patterns.

[Search image of the control unit (automatic offload function unit) 11 using Simple GA]
FIG. 3 is a diagram showing a search image of the process of the control unit (automatic offload function unit) 11 by the Simple GA and a gene sequence mapping of a for statement.
GA is a combinatorial optimization method that mimics the evolutionary process of living organisms. The GA flowchart is as follows: Initialization → Evaluation → Selection → Crossover → Mutation → Termination decision.
In this embodiment, a simple GA with simplified processing is used among GAs. The simple GA is a simplified GA in which genes are only 1 and 0, and roulette selection, one-point crossover, and mutation reverse the value of one gene.

<Initialization>
In the initialization, after checking whether all for statements in the application code can be parallelized, parallelizable for statements are mapped to gene arrays. If GPU processing is used, it is set to 1, and if not, it is set to 0. A specified number of individuals M is prepared for genes, and 1 or 0 is randomly assigned to each for statement.
Specifically, the control unit (automatic offload function unit) 11 (see FIG. 1) acquires the application code 125 (see FIG. 2) used by the user, and checks whether or not for statements can be parallelized from the code patterns 141 of the application code 125, as shown in FIG. 3. As shown in FIG. 3, if three for statements are found from the code pattern 141 (see symbol b in FIG. 3), one digit is assigned to each for statement, and three digits of 1 or 0 are assigned to the three for statements in this case. For example, 0 is assigned when processing is performed by the CPU, and 1 is assigned when output to the GPU. However, 1 or 0 is assigned randomly at this stage.
The code corresponding to the gene length is three digits, and the three-digit gene length code has ^2-3 patterns, for example, 100, 101, .... In Fig. 3, the circles (○ marks) in the code pattern 141 are shown as an image of the code.

<Evaluation>
In the evaluation, deployment and performance measurement are performed (see symbol c in FIG. 3). That is, the performance measurement unit 118 (see FIG. 1) compiles the code corresponding to the gene, deploys it on the verification machine 14, and executes it. The performance measurement unit 118 performs benchmark performance measurement. The degree of suitability of genes of patterns with good performance (parallel processing patterns) is increased.

<Select>
In the selection, high performance code patterns are selected based on the fitness (see symbol d in FIG. 3). The performance measurement unit 118 (see FIG. 1) selects a specified number of genes with high fitness based on the fitness. In this embodiment, roulette selection according to the fitness and elite selection of the genes with the highest fitness are performed.
FIG. 3 shows, as a search image, that the number of circles (◯) in Select code patterns 142 has been reduced to three.

<Crossover>
In crossover, some genes are exchanged at a certain point between selected individuals at a certain crossover rate Pc to create offspring individuals.
The genes of a certain pattern (parallel processing pattern) selected by roulette wheel are crossed with those of another pattern. The position of the one-point crossover is arbitrary, and for example, the crossover is performed at the second digit of the above three-digit code.

<Mutation>
In order to avoid local solutions, mutation is introduced. However, in order to reduce the amount of calculation, mutation may not be performed. In mutation, each value of an individual's genes is changed from 0 to 1 or from 1 to 0 at a certain mutation rate Pm.

<End Judgment>
As shown in FIG. 3, next generation code patterns after crossover & mutation are generated (see symbol e in FIG. 3).
In the termination determination, the process is terminated after repeating the process for a specified number of generations T, and the gene with the highest fitness is taken as the solution.
For example, performance is measured and the three fastest ones, 101, 010, and 001, are selected. These three are then recombined in the next generation using GA to create a new pattern (parallel processing pattern) 101 (one example), for example. At this time, a mutation is automatically introduced into the recombined pattern, such as changing 0 to 1. The above is repeated to find the fastest pattern. A designated generation (for example, the 20th generation) is decided, and the pattern remaining in the final generation is designated as the final solution.

<Deployment>
The parallel processing pattern with the highest processing performance that corresponds to the gene with the highest fitness is then redeployed into a production environment and provided to the user.

We will explain the case where there are a considerable number of for statements (loop statements; repetitive statements) that cannot be offloaded to the GPU. For example, even if there are 200 for statements, only about 30 can be offloaded to the GPU. Here, we will exclude those that will result in an error, and run GA on these 30 statements.

For example, OpenACC, a specification for GPU processing of C/C++ code, has a compiler that allows you to specify the directive #pragma acc kernels to extract bytecode for the GPU and execute it to enable GPU offloading. In the case of Python and Java, you can specify GPU processing using CUDA or Java Lambda expressions.

Also, when C/C++ is used, the C/C++ code is analyzed to find for statements. When a for statement is found, #pragma acc kernels, which is a parallel processing syntax in OpenACC, is used to write to the for statement. In detail, for statements are inserted one by one into an empty #pragma acc kernels and compiled. If an error occurs, the for statement is excluded since it cannot be processed by the GPU in the first place. In this way, the remaining for statements are found. The one that does not produce an error is taken as the length (gene length). If there are five for statements without errors, the gene length is 5, and if there are 10 for statements without errors, the gene length is 10. Note that parallel processing is not possible when there is a data dependency such that the previous processing is used for the next processing.
The above is the preparation stage. Next, the GA process is carried out.

A code pattern with a gene length corresponding to the number of for statements is obtained. First, parallel processing patterns 100, 010, 001, ... are randomly assigned. Then, GA processing is performed and compilation is performed. At this time, an error may occur even though the for statements can be offloaded. This is the case when the for statements are hierarchical (GPU processing is possible if either one is specified). In this case, the for statement that caused the error can be left. Specifically, there is a method of making it take longer to process, causing a timeout.

Deploy it on the verification machine 14 and benchmark it - for example, if it is image processing, benchmark it with that image processing, and evaluate the adaptability as higher as the processing time is shorter. For example, the reciprocal of the processing time, 1 is given for a processing time of 10 seconds, 0.1 for a processing time of 100 seconds, and 10 for a processing time of 1 second.
Those with high adaptability are selected, for example, 3 to 5 out of 10 are selected, and a new code pattern is created by rearranging them. During the creation process, it is possible that the same code pattern as before is created. In this case, since it is not necessary to perform the same benchmark, the same data as before is used. In this embodiment, the code pattern and its processing time are stored in the storage unit 13.

The above explains the search image of the control unit (automatic offload function unit) 11 using Simple GA.

A method for offloading loop statements of application software to a GPU will be described below.
[Flowchart for offloading loop statements]
4A and 4B are flowcharts of the offloading of loop statements by the offload server 1, and FIGS. 4A and 4B are connected by a connector.

<Receive application code and operating conditions>
In step S100, the control unit 11 (see FIG. 1) receives an application code and operation conditions. The operation conditions are, for example, user operation conditions, and the control unit 11 (see FIG. 1) acquires the operation conditions by referring to information in the facility resource DB 132 (see FIG. 1).

<Code Analysis>
In step S101, the application code analysis unit 112 (see FIG. 1) performs a code analysis of the application program.

<Loop statement identification>
In step S102, the parallel processing specification unit 114 (see FIG. 1) identifies loop statements and reference relationships in the application program.

<Loop statement loop count>
In step S103, the parallel processing specification unit 114 runs a benchmark tool to grasp the number of loops of the loop statement and allocate a threshold value.

<Parallel processing of loop statements>
In step S104, the parallel processing specification unit 114 checks whether each loop statement can be processed in parallel.

<Repeat loop statement>
The control unit (automatic offload function unit) 11 repeats the processing of steps S106-S107 between the loop start point of step S105 and the loop end point of step S108 for the number of loop statements.
In step S106, the parallel processing specification unit 114 specifies GPU processing for each loop statement using a language-specific method, and compiles or interprets the loop statement.
In step S107, if an error occurs, the parallel processing designation unit 114 deletes the GPU processing designation from the corresponding for statement.
In step S109, the parallel processing designation unit 114 counts the number of for statements for which no compilation error occurs, and sets the count as the gene length.

<Preparation of designated population patterns>
Next, the parallel processing designation unit 114 prepares a designated number of gene sequences as initial values. Here, these are created by randomly assigning 0 and 1.
In step S110, the parallel processing designation unit 114 maps the code of the application program to genes. A designated population pattern is prepared by mapping a gene sequence in which 0s and 1s are randomly assigned to genes.
According to the prepared gene sequence, a directive that specifies parallel processing when the gene value is 1 is inserted into the code of the application program (see, for example, the #pragma directive in FIG. 3).

The control unit (automatic offload function unit) 11 repeats the processing of steps S112-S119 for a designated number of generations between the loop start point of step S111 and the loop end point of step S120.
In addition, in the above-mentioned repetition of the specified number of generations, the processing of steps S113-S116 is further repeated a specified number of individuals between the loop start point of step S112 and the loop end point of step S117. That is, within the repetition of the specified number of generations, the repetition of the specified number of individuals is processed in a nested state.

<Data transfer specification>
In step S113, the data transfer specification unit 113 specifies a data transfer based on the variable reference relationship using a method appropriate for the language.

<Compile>
In step S114, the parallel processing pattern creation unit 117 (see FIG. 1) compiles or interprets the code with directives specified according to the gene pattern on the GPU processing platform. That is, the parallel processing pattern creation unit 117 compiles or interprets the created application program code with a PGI compiler on the verification machine 14 equipped with a GPU.
Here, a compilation error may occur when multiple nested for statements are specified in parallel, etc. In this case, it is treated the same as when the processing time during performance measurement times out.

In step S115, the performance measurement unit 118 (see FIG. 1) deploys the executable file to the verification machine 14 equipped with a CPU and GPU.
In step S116, the performance measurement unit 118 executes the arranged binary file and measures the benchmark performance when offloaded.

Here, in the intermediate generation, genes with the same pattern as before are not measured, and the same values are used. In other words, if a gene with the same pattern as before is generated during the GA process, compilation or performance measurement is not performed for that individual, and the same measured value as before is used.
In step S118, the executable file creation unit 119 (see FIG. 1) evaluates the individuals with shorter processing times so that they have a higher fitness, and selects the individuals with the highest performance.

In step S119, the executable file creation unit 119 performs crossover and mutation processes on the selected individuals to create the next generation individuals. The next generation individuals are subjected to compilation, performance measurement, fitness setting, selection, crossover, and mutation processes.
That is, after benchmark performance is measured for all individuals, the fitness of each gene sequence is set according to the benchmark processing time. Individuals to be left are selected according to the set fitness. The selected individuals are subjected to GA processing including crossover, mutation, and direct copy to create the next generation population.

In step S121, after the GA processing for the specified number of generations is completed, the executable file creation unit 119 determines that the C/C++ code corresponding to the highest-performance gene sequence (the highest-performance parallel processing pattern) is the solution.

<GA parameters>
The above-mentioned number of individuals, number of generations, crossover rate, mutation rate, fitness setting, and selection method are parameters of the GA. The parameters of the GA may be set, for example, as follows.
The parameters and conditions for the Simple GA to be executed can be, for example, as follows:
Gene length: Number of loop statements that can be parallelized Number of individuals M: Less than or equal to gene length Number of generations T: Less than or equal to gene length Fitness: (processing time) ^-1/2
With this setting, the shorter the benchmark processing time, the higher the fitness. Also, by setting the fitness to (processing time) ^-1/2 , it is possible to prevent the fitness of a specific individual with a short processing time from becoming too high, narrowing the search range. Also, if the performance measurement does not end within a certain time, a timeout is set and the fitness is calculated assuming a processing time of 1000 seconds or the like (long time). This timeout time can be changed according to the performance measurement characteristics.
Selection: Roulette selection However, elite preservation is also performed in which the genes with the highest fitness in one generation are preserved in the next generation without crossover or mutation.
Crossover rate Pc: 0.9
Mutation rate Pm: 0.05

In this embodiment, applications that have many loops and take a long time to execute are identified in advance using gcov, gprof, etc., and offloading is attempted. This makes it possible to find applications that can be accelerated efficiently.

In order to search for the offloaded part in a shorter time, it is possible to measure the performance in parallel for the number of individuals on multiple verification machines. Adjusting the timeout time according to the application program can also shorten the time. For example, if the offload processing takes twice as long as the execution time on the CPU, a timeout can be set. In addition, the more individuals and generations there are, the higher the chance of finding a high-performance solution. However, when maximizing each parameter, compilation and performance benchmarking must be performed for the number of individuals x the number of generations. This takes time before the actual service can be used. In this embodiment, the GA is performed with a small number of individuals and generations, but by setting the crossover rate Pc to a high value of 0.9 and searching a wide range, a solution with a certain level of performance can be found quickly.

[Flowchart for setting resource ratios and resource amounts]
Fig. 5 is a flowchart for setting the resource ratio and resource amount added after a GPU or FPGA offload attempt. The flowchart shown in Fig. 5 is executed after a GPU offload attempt shown in Figs. 4A-B, and also after a GPU or FPGA offload attempt shown in Figs. 7 and 8 described below.

In step S51, the resource ratio determination unit 115 acquires the user operation conditions, the test case CPU processing time, and the offload device processing time. The user operation conditions are specified by the user when specifying the code that the user wants to offload. The resource amount setting unit 116 uses the user operation conditions when determining the resource amount by referring to the information in the equipment resource DB 132.

<Processing of resource ratio determination unit 115>
In step S52, the resource ratio determination unit 115 determines the ratio between the processing time of the CPU and the offload device (the test case CPU processing time and the offload device processing time) as a resource ratio based on the performance measurement result.

As described in Figures 3 and 4, this automatic offloading already obtains performance measurement results in the verification environment when converting code. Using these performance measurement results, the resource ratio determination unit 115 determines the resource ratio between the CPU and the offload device. Specifically, it determines an appropriate resource ratio for the ratio of the processing times of the CPU and the offload device in the verification environment. For example, if the test case processing time in the verification environment is CPU processing: 10 seconds, GPU processing: 5 seconds, the resource ratio will be CPU:GPU = 2:1.

The resource ratio determination unit 115 determines the resource ratio so that the processing times of the CPU and the offload device are of the same order. By determining the resource ratio so that the processing times of the CPU and the offload device are of the same order, the processing times of the CPU and the offload device are aligned, and the resource amount can be appropriately set even in a mixed environment where the CPU and accelerator are GPU, FPGA, many-core CPU, etc.

If the difference in processing time between the CPU and the offload device is equal to or greater than a predetermined threshold, the resource ratio determination unit 115 sets the resource ratio to a predetermined upper limit. In other words, if the difference between the processing time of the CPU and the offload device in the verification environment is, for example, 10 times or more, setting the resource ratio to 10 times or more will lead to a deterioration in cost performance. In this case, for example, the resource ratio is set to an upper limit of 5:1 (the upper limit is a resource ratio of 5:1 processing time). By setting an upper limit on the resource ratio, a significant increase in the number of VMs can be prevented.

<Processing of resource amount setting unit 116>
In step S53, the resource amount setting unit 116 sets the resource amount based on the user operation conditions and the appropriate resource ratio. That is, the resource amount setting unit 116 determines the resource amount while keeping the resource ratio as much as possible so as to satisfy the cost conditions specified by the user.

The resource amount setting unit 116 maintains an appropriate resource ratio and sets the maximum resource amount that satisfies the user's operational conditions. As a specific example, assume that a CPU 1VM costs 1,000 yen/month, a GPU costs 4,000 yen/month, a resource ratio of 2:1 is appropriate, and the user has a monthly budget of 10,000 yen or less. In this case, 2 CPUs and 1 GPU are secured and placed in a commercial environment.

If the minimum resource amount while maintaining the resource ratio does not satisfy the user operation conditions, the resource amount setting unit 116 breaks the resource ratio and sets the resource amount of the CPU and offload device to satisfy the cost conditions. As a specific example, assume that the appropriate resource ratio is 2:1, with 1 VM CPU costing 1,000 yen/month and 4,000 yen/month GPU, and that the user has a monthly budget of 5,000 yen or less. In this case, since the user budget is insufficient, the resource ratio cannot be maintained, but the resource amount of the CPU and offload device is set smaller, i.e., 1 CPU and 1 GPU are secured and arranged.

After completing the processing of step S53 above and reserving and placing resources in the commercial environment, the automatic verification described in Figure 2 is executed to check performance and cost before the user uses it. This allows resources to be reserved in the commercial environment, and after automatic verification, the performance and cost can be presented to the user.

<Summary of resource ratio and resource amount settings>
In order to optimize the resource ratio, the performance measurement results when determining the offload pattern solution are used. In the implementation, the resource ratio is determined based on the processing time of the test case so that the CPU and GPU processing times are of the same order of magnitude. For example, if the processing time of the test case is 10 seconds for CPU processing and 5 seconds for GPU processing, the CPU side resources are considered to be doubled and the processing time is approximately the same, so the resource ratio is 2:1. Note that since the number of virtual machines, etc. is an integer, the resource ratio is rounded off to an integer ratio when calculated from the processing time.

Once the resource ratio has been determined, the next step is to set the amount of resources to be used when deploying applications in a commercial environment. In determining the amount of resources, the implementation determines the number of VMs, etc., while keeping the resource ratio as much as possible so as to satisfy the cost requirements specified by the user when requesting offloading. Specifically, the maximum number of VMs, etc. is selected while keeping the resource ratio within the cost range.

For example, for the CPU, 1 VM is 1000 yen/month, 4000 yen/month for the GPU, and a resource ratio of 2:1 is appropriate, and if the user has a monthly budget of 10,000 yen or less, 2 CPUs and 1 GPU are secured. If the resource ratio cannot be kept within the cost range, the resource amount is set so as to get as close as possible to the appropriate resource ratio, starting with 1 CPU and 1 GPU. For example, if the monthly budget is 5,000 yen or less, the resource ratio cannot be kept, but 1 CPU and 1 GPU are secured.
When the resource amount is set, the implementation allocates CPU and GPU resources using, for example, the virtualization function of Xen Server.

Second Embodiment
The present invention is characterized by the setting of resource ratios and resource amounts in automatic offloading.
In the second embodiment, the setting of resource ratios and resource amounts in function block offloading will be described. First, the function block offloading will be described.
The overall configuration and operation of the function block offload will be described with reference to FIGS.

6 is a functional block diagram showing a configuration example of an offload server 200 according to the second embodiment of the present invention. The same components as those in FIG. 1 are denoted by the same reference numerals, and the description of the overlapping parts will be omitted.
The offload server 200 is a device that automatically offloads specific processing of an application program to an accelerator.

As shown in FIG. 6, the offload server 200 includes a control unit 210, an input/output unit 12, a memory unit 130, and a verification machine 14 (accelerator verification device).

The input/output unit 12 is composed of a communication interface for transmitting and receiving information between each device, etc., and an input/output interface for transmitting and receiving information between input devices such as a touch panel or keyboard, and output devices such as a monitor.

The memory unit 130 is composed of a hard disk, flash memory, RAM, etc., and temporarily stores programs (offload programs) for executing each function of the control unit 210, and information necessary for processing by the control unit 210 (e.g., intermediate language file 133).

The memory unit 13 includes a code pattern database 230 (described below), a test case DB 131, an equipment resource DB 132, and an intermediate language file 133.

Test case DB131 stores performance test items. Test case DB131 stores information for conducting tests to measure the performance of applications that are being accelerated. For example, in the case of a deep learning application for image analysis processing, the test items are sample images and the test items for executing them.

The verification machine 14 is equipped with a CPU (Central Processing Unit), GPU, and FPGA as a verification environment for the environment-adaptive software.

<Code Pattern DB230>
Storage of GPU libraries and IP cores The code pattern DB 230 stores libraries and IP cores (described later) that can be offloaded to a GPU, FPGA, etc. That is, for <Process B-1> described later, the code pattern DB 230 stores specific libraries, libraries for GPUs (GPU libraries) that speed up function blocks, IP cores for FPGAs (IP cores), and related information. For example, the code pattern DB 230 stores a library list (external library list) for arithmetic calculations such as FFT, etc.

Storage of CUDA library The code pattern DB 230 stores, as a GPU library, for example, a CUDA library and a library usage procedure for using the CUDA library. That is, in <Process C-1> described later, when a library or IP core to be replaced is implemented in a GPU or FPGA and connected to a host side (CPU) program, the library usage procedure is also registered and used according to the procedure. For example, in the CUDA library, the procedure for using the CUDA library from C language code is published together with the library, so the library usage procedure is also registered in the code pattern DB 230.

Storage of classes and structures The code pattern DB 230 stores classes or structures of processes that will be written in a similar manner when calculated on the host. That is, in <Process B-2> described later, in order to detect functional processes other than unregistered library calls, classes, structures, etc. are detected from the definition description of the source code by syntax analysis. For <Process B-2> described later, the code pattern DB 230 registers classes or structures of processes that will be written in a similar manner when calculated on the host. The existence of libraries or IP cores that speed up the functional processes of the classes or structures is detected by a similarity detection tool (described later).

Storage of OpenCL code The code pattern DB 230 stores OpenCL code as information related to IP cores. By storing the OpenCL code in the code pattern DB 230, it is possible to connect a CPU and an FPGA using an OpenCL interface and to implement an IP core in an FPGA from the OpenCL code via a high-level synthesis tool (described later) of an FPGA vendor such as Xilinx or Intel.

<Control Unit 210>
The control unit 210 is an automatic offloading function that controls the entire offload server 200, and is realized by a CPU (not shown) that expands a program (offload program) stored in the memory unit 130 into RAM and executes it.

In particular, the control unit 210 detects functional blocks in existing program code for the CPU that can be offloaded to an FPGA or GPU to speed up processing, and performs offload processing of the functional blocks to speed up processing by replacing the detected functional blocks with libraries for the GPU or IP cores for the FPGA, etc.

The control unit 210 includes an application code specification unit 111, an application code analysis unit 112, a replacement function detection unit 213, a replacement processing unit 214, a resource ratio determination unit 115, a resource amount setting unit 116, an offload pattern creation unit 215, a performance measurement unit 118, an executable file creation unit 119, a production environment deployment unit 120, a performance measurement test extraction and execution unit 121, and a user provision unit 122.

<Application code specification unit 111>
The application code designation unit 111 designates the input application code. Specifically, the application code designation unit 111 passes the application code described in the received file to the application code analysis unit 112.

<Application Code Analysis Unit 112>
In <Process A-1> described later, the application code analysis unit 112 analyzes the source code of the application program and detects calls to external libraries contained in the source code. Specifically, the application code analysis unit 112 uses a syntax analysis tool such as Clang to analyze the source code to analyze library calls and functional processing contained in the code, along with loop statement structures, etc.

The code analysis described above is difficult to generalize, as it requires analysis that takes into account the device to be offloaded. However, it is possible to grasp the code structure, such as loop statements and variable reference relationships, and to determine that a functional block performs FFT processing and that it calls a library that performs FFT processing. It is difficult for the offload server to automatically determine the functional block. This can also be grasped by using a similarity detection tool such as Deckard to determine the similarity. Here, Clang is a tool for C/C++, but it is necessary to choose a tool that suits the language to be analyzed.

In addition, in <Process A-2> described below, the application code analysis unit 112 detects class or structure code from the source code.

<Replacement function detection unit 213>
In <Process B-1> described later, the replacement function detection unit 213 uses the detected call as a key to acquire a GPU library and an IP core from the code pattern DB 230. Specifically, the replacement function detection unit 213 compares the detected library call with the code pattern DB 230 using the library name as a key to extract offloadable processing that can be offloaded to a GPU or FPGA.

Here, the code pattern DB 230 stores, as a GPU library, for example, a CUDA library and a library usage procedure for using the CUDA library. The replacement function detection unit 213 then retrieves the CUDA library from the code pattern DB 230 based on the library usage procedure.

In <Process B-2> described below, the replacement function detection unit 213 uses the definition description code of the detected class or structure (described below) as a key to obtain a GPU library and an IP core from the code pattern DB 230. Specifically, the replacement function detection unit 213 uses a similarity detection tool that detects copied code and definition description code that has been changed after copying to extract GPU libraries and IP cores that can be offloaded to a GPU or FPGA that are linked to a similar class or structure from the code pattern DB 230 for the class or structure included in the source code to be replaced.

<Replacement Processing Unit 214>
In <Processing C-1> described later, the replacement processing unit 214 replaces the processing description of the source of the application program source code with the processing description of the library and IP core of the replacement destination acquired by the replacement function detection unit 213. Specifically, the replacement processing unit 214 replaces the extracted offloadable processing with a library for GPU, an IP core for FPGA, or the like.
The replacement processing unit 214 also offloads the processing description of the replaced library and IP core to a GPU, FPGA, or the like as a functional block to be offloaded. Specifically, the replacement processing unit 214 offloads the functional block replaced by a library for a GPU or an IP core for an FPGA, or the like, by creating an interface with the CPU program. The replacement processing unit 214 outputs an intermediate language file 133 such as CUDA or OpenCL.

In the process C-2 described below, the replacement processing unit 214 replaces the source processing description of the application program source code with the processing description of the acquired library and IP core, and notifies the user of confirmation if the number or type of arguments or return values differs between the source and destination.

<Offload Pattern Creation Unit 215>
The offload pattern creation unit 215 creates one or more offloading patterns. Specifically, an interface with the host program is created, and the offloading and non-offloading patterns are tried through performance measurement in a verification environment, thereby extracting an offloading pattern that provides a higher speed.

Here, the code pattern DB 230 stores OpenCL code as information related to IP cores. When offloading to a PLD such as an FPGA, the offload pattern creation unit 215 connects the host and the PLD using an OpenCL interface based on the OpenCL code, and implements the IP core in the PLD based on the OpenCL code.

This section describes the interface description between the kernel program and the host program in accordance with the OpenCL API. Note that the following explanation corresponds to a specific example of <Process C-1> in [Process C] (matching the interface with the host side) described below.

A kernel created according to the OpenCL C language syntax is executed on a device (e.g., FPGA) by a program created on the host (e.g., CPU) side using the OpenCL C language runtime API. The part where the kernel function hello() is called from the host side is to call clEnqueueTask(), which is one of the OpenCL runtime APIs.
The basic flow of OpenCL initialization, execution, and termination described in the host code consists of the following steps 1 to 13. Of these steps 1 to 13, steps 1 to 10 are the procedure (preparation) until the kernel function hello() is called from the host side, and step 11 is the execution of the kernel.

1. Platform Specific
The platform on which OpenCL runs is identified using a function clGetPlatformIDs( ) that provides platform identification functionality defined in the OpenCL runtime API.

2. Device Identification
A device such as a GPU to be used on the platform is identified using a function clGetDeviceIDs( ) that provides a device identification function defined in the OpenCL runtime API.

3. Creating a context
An OpenCL context that serves as an execution environment for running OpenCL is created using a function clCreateContext( ) that provides a context creation function defined in the OpenCL runtime API.

4. Creating a command queue
A command queue is created in preparation for controlling the device using the function clCreateCommandQueue(), which provides a command queue creation function defined in the OpenCL runtime API. In OpenCL, the host issues commands to the device (such as issuing kernel execution commands and memory copy commands between the host and device) through the command queue.

5. Creating a memory object
A memory object that allows the host side to reference the memory object is created using a function clCreateBuffer( ) that provides a function for allocating memory on a device defined in the OpenCL runtime API.

6. Loading a kernel file The execution of the kernel executed on the device is controlled by the host program. For this reason, the host program must first load the kernel program. The kernel program includes binary data created by the OpenCL compiler and source code written in the OpenCL C language. This kernel file is loaded (description omitted). Note that the OpenCL runtime API is not used when loading the kernel file.

7. Creating a program object
In OpenCL, a kernel program is recognized as a program object. This procedure is called program object creation.
Create a program object that allows the host to reference the memory object by using the function clCreateProgramWithSource(), which provides the program object creation function defined in the OpenCL runtime API. When creating a program object from a compiled binary sequence of a kernel program, use clCreateProgramWithBinary().

8. Build Build the program object registered as source code using the OpenCL C compiler and linker.
A program object is built using the function clBuildProgram(), which executes a build by the OpenCL C compiler and linker defined in the OpenCL runtime API. Note that if a program object is generated from a compiled binary string using clCreateProgramWithBinary(), this compilation procedure is not necessary.

9. Creating a kernel object
A kernel object is created using the function clCreateKernel(), which provides a kernel object creation function defined in the OpenCL runtime API. Since one kernel object corresponds to one kernel function, the name of the kernel function (hello) is specified when creating the kernel object. Furthermore, if multiple kernel functions are written as one program object, one kernel object corresponds one-to-one to one kernel function, so clCreateKernel() is called multiple times.

10. Kernel argument settings
The kernel arguments are set using the function clSetKernel(), which provides a function for giving arguments to the kernel defined in the OpenCL runtime API (passing values to arguments held by the kernel function).
As described above, the preparation is completed in steps 1 to 10, and the process advances to step 11 in which the kernel is executed on the device from the host side.

11. Kernel Execution Kernel execution (submission to the command queue) is an action on the device, so it is a queuing function for the command queue.
A command to execute the kernel hello on the device is queued using a function clEnqueueTask() that provides a kernel execution function defined in the OpenCL runtime API. After the command to execute the kernel hello is queued, it will be executed on an executable computing unit on the device.

12. Reading from a memory object
Data is copied from the device memory area to the host memory area using the function clEnqueueReadBuffer(), which provides a function for copying data from device memory to host memory defined in the OpenCL runtime API. Data is also copied from the host memory area to the device memory area using the function clEnqueueWrightBuffer(), which provides a function for copying data from the host memory to the device memory. Note that these functions act on the device, so the copy command is queued in the command queue once before data copying begins.

13. Releasing objects Finally, release the various objects that you have created up to this point.
The above describes device execution of a kernel written in accordance with the OpenCL C language.

<Performance Measuring Unit 118>
The performance measurement unit 118 compiles an application program of the created processing pattern, places it on the verification machine 14, and executes processing for performance measurement when offloaded to a GPU, FPGA, or the like.
The performance measurement unit 118 includes a binary file deployment unit 118a that deploys binary files derived from the intermediate language on the verification machine 14 that includes a GPU and an FPGA.

The performance measurement unit 118 executes the placed binary file, measures the performance when offloaded, and returns the performance measurement results to the binary file placement unit 118a. In this case, the performance measurement unit 118 attempts to measure performance based on the extracted intermediate language using another extracted processing pattern.

A specific example of performance measurement will be described.
The offload pattern creation unit 215 creates a processing pattern for offloading a functional block that can be offloaded to a GPU or FPGA, and the intermediate language of the created processing pattern is compiled by the executable file creation unit 119. The performance measurement unit 118 measures the performance of the compiled program ("first performance measurement").

The offload pattern creation unit 215 then creates a list of processing patterns that have been measured and have higher performance than the CPU. The offload pattern creation unit 215 creates a new processing pattern for offloading by combining the processing patterns in the list. The offload pattern creation unit 215 creates the combined offload processing pattern and an intermediate language, and the executable file creation unit 119 compiles the intermediate language.
The performance measurement unit 118 measures the performance of the compiled program ("second performance measurement").

<Executable File Creation Unit 119>
The executable file creation unit 119 compiles the intermediate language of the processing pattern to be offloaded to create an executable file. Based on the performance measurement results obtained by repeating a certain number of times, the executable file creation unit 119 selects the processing pattern with the highest processing performance from one or more processing patterns, compiles the processing pattern with the highest processing performance, and creates a final executable file.

Below, we will explain the functional block offload processing of the offload server 200 configured as described above.

The outline of the above-mentioned function block offloading process and points to consider will be described below.
With regard to FPGAs, since hardware circuit design can take a lot of time, functions that have already been designed are often made reusable in the form of IP cores (Intellectual Property Cores). IP cores are partial circuit information for configuring semiconductors such as FPGAs, ICs, and LSIs, and are organized by function. Typical examples of IP core functions include encryption/decryption processing, arithmetic operations such as FFT (Fast Fourier Transform), image processing, and audio processing. Many IP cores require a license fee, but some are provided free of charge.

In the second embodiment, for FPGAs, IP cores are used for automatic offloading. For GPUs, although they are not called IP cores, FFT and linear algebra operations are typical examples of functions, and cuFFT and cuBLAS implemented using CUDA are provided free of charge as libraries for GPUs. In this second embodiment, these libraries are utilized for GPUs.

In the second embodiment, when existing program code created for a CPU contains functional blocks such as FFT processing that can be accelerated by offloading to a GPU or FPGA, the code is accelerated by replacing the functional blocks with GPU libraries or FPGA IP cores.

[Function block offload processing details]
For offloading of a functional block, three elements must be considered: detection of the functional block (hereinafter referred to as "Process A"), detection of whether the functional block has an existing library/IP core, etc. for offloading (hereinafter referred to as "Process B"), and consistency of the interface with the host side when replacing the functional block with the library/IP core, etc. (hereinafter referred to as "Process C"). The offloading process of the functional block will be described in detail according to the considerations of the above three elements.

[Process A] (detection of functional blocks)
"Processing A" (detection of function blocks) is divided into <Processing A-1>, which calls a function in a library and makes the function call in the library a function block, and <Processing A-2>, which detects a class, structure, etc. and makes the function block if the function call is in an unregistered library. That is, <Processing A-1> detects a function call in an existing library and makes it a function block, and <Processing A-2> extracts a function block from a class or structure if a function block is not detected in <Processing A-1>.

<Process A-1>
The application code analysis unit 112 uses syntax analysis to detect whether a function of an external library is being called from source code. The details are as follows. The code pattern DB 230 holds a library list for arithmetic calculations such as FFT. The application code analysis unit 112 performs syntax analysis of the source code, and compares it with the library list held by the code pattern DB 230 to detect whether a function of an external library is being called.

<Process A-2>
The application code analysis unit 112 detects the functional processing of a class or structure from the definition description of the source code using syntax analysis in order to detect functional processing other than unregistered library calls as functional blocks. The application code analysis unit 112 detects, for example, a structure that is a type that groups together several variables defined using the struct of the C language, and a class that is a reference type for a structure whose instantiated object type is a value type. The application code analysis unit 112 also detects a class that is used as an alternative to a structure in Java (registered trademark), for example.

[Process B] (Detection of offloadable functions)
[Processing B] (detection of offloadable functions) is divided into <Processing B-1>, which receives <Processing A-1> and refers to the code pattern DB 230 to obtain replaceable GPU libraries and IP cores, and <Processing B-2>, which receives <Processing A-2> and replaces the processing description of the application code to be replaced with the GPU library and IP core processing description to be replaced. That is, <Processing B-1> obtains replaceable GPU libraries and IP cores from the code pattern DB 230 using the library name as a key. <Processing B-2> detects replaceable GPU libraries and IP cores using codes such as classes and structures as a key, and replaces the processing description of the application code to be replaced with the GPU library and IP core processing description to be replaced.

As a premise for process B, the code pattern DB 230 holds a specific library, a library for GPUs that speeds up a function block, an IP core for FPGAs, and related information. In addition, the code pattern DB 230 registers code and executable files along with function names for the original libraries and function blocks.

<Process B-1>
The replacement function detection unit 213 searches the code pattern DB 230 using the library name as a key for the library call detected by the application code analysis unit 112 in <Process A-1>, and obtains a replaceable GPU library (a library for GPUs that can be accelerated) and an IP core for FPGA from the code pattern DB 230.

An example of <Process B-1> is described below.
For example, if the processing to be replaced is 2D FFT processing, the replacement function detection unit 213 detects OpenCL code as FPGA processing for processing 2D FFT using the external library name as a key (host program, kernel program, etc.). The OpenCL code is stored in the code pattern DB 230.

For example, if the processing to be replaced is a 2D FFT processing, the replacement function detection unit 213 replaces it with a function call in cuFFT detected as a GPU library. The GPU library is stored in the code pattern DB 230.

<Process B-2>
The replacement function detection unit 213 searches the code pattern DB 230 using the code of the class, structure, etc. detected by the application code analysis unit 112 in <Process A-2> as a key, and obtains a replaceable GPU library (a library for GPU that can be accelerated) and an IP core for FPGA from the code pattern DB 230 using a similarity detection tool. The similarity detection tool is a tool that detects copied code and code that has been changed after copying, such as Deckard. By using the similarity detection tool, the replacement function detection unit 213 can detect some processes that are written in a similar manner when calculated by a CPU, such as matrix calculation code, and processes that have been copied and changed by copying someone else's code. Note that the similarity detection tool does not target classes that have been newly created independently, as it is difficult to detect them.

An example of <Process B-2> is described below.
The replacement function detection unit 213 searches for similar classes and structures registered in the code pattern DB 230 for classes and structures detected in the source CPU code using a similarity detection tool such as Deckard. For example, if the source process (whose code is in Non-Patent Document 4, etc.) is a 2D FFT class, the class registered in the code pattern DB 230 as its similar class is the 2D FFT class. The code pattern DB 230 registers IP cores and GPU libraries capable of offloading 2D FFT. Therefore, similar to <Process B-1>, OpenCL code (host program, kernel program, etc.) and GPU libraries are detected for 2D FFT.

[Process C] (Interface matching with the host side)
[Processing C] (matching the interface with the host side) includes <Processing C-1> and <Processing C-2>. <Processing C-1> receives <Processing B-1>, and replaces the processing description of the application code to be replaced with the GPU library and IP core processing description of the replacement destination, and also describes interface processing for calling the GPU library and IP core. <Processing C-2> receives <Processing B-2>, and replaces the processing description of the application code to be replaced with the GPU library and IP core processing description of the replacement destination, and also describes interface processing for calling the GPU library and IP core. Here, the description of the interface processing for calling the GPU library and IP core corresponds to "matching the interface with the host side".

<Process C-1>
The replacement processing unit 214 replaces the source processing description of the application code with the destination GPU library and IP core processing description. The replacement processing unit 214 then writes an interface process for calling the GPU library and IP core (OpenCL API, etc.), and compiles the created pattern.

<Process C-1> will now be described in more detail.
The replacement function detection unit 213 searches for a corresponding library or IP core in <Process B-1> for the library call detected in <Process A-1>. For this reason, the replacement processing unit 214 implements the library or IP core to be replaced in the GPU or FPGA, and performs interface processing to connect it to the host side (CPU) program.

In the case of a GPU library, a library such as CUDA is assumed, and a method for using the CUDA library from C language code is published together with the library. Therefore, the library usage method is also registered in the code pattern DB 230, and the replacement processing unit 214 replaces the source processing description of the application code with the destination GPU library according to the library usage method registered in the code pattern DB 230, and performs predetermined descriptions such as calling functions to be used in the GPU library.

In the case of an IP core for FPGA, HDL (Hardware Description Language) and the like are assumed. In this case, OpenCL code is also stored in the code pattern DB 230 as information related to the IP core. The replacement processing unit 214 can perform interface processing with the FPGA via a high-level synthesis tool (e.g., Xilinx Vivado, Intel HLS Compiler, etc.). The replacement processing unit 214 performs connection between the CPU and the FPGA using an OpenCL interface from the OpenCL code via the high-level synthesis tool. Similarly, the replacement processing unit 214 implements the IP core in the FPGA via a high-level synthesis tool of an FPGA vendor such as Xilinx or Intel.

<Process C-2>
The replacement processing unit 214 replaces the processing description of the application code to be replaced with the GPU library and IP core processing description to be replaced. If the number and type of arguments and return values are different between the source and destination, the replacement processing unit 214 checks with the user, describes interface processing for calling the GPU library and IP core (OpenCL API, etc.), and compiles the created pattern. That is, in <Process C-2>, the replacement processing unit 214 searches for libraries and IP cores that can be accelerated in <Process B-2> for the classes, structures, etc. detected in <Process A-2>. Therefore, in <Process C-2>, the replacement processing unit 214 implements the corresponding libraries and IP cores in the GPU or FPGA.

<Process C-2> will now be described in more detail.
In <Process C-1>, because it is a library or IP core that speeds up a specific library call, it is necessary to generate an interface, but the number and type of arguments and return values expected by the GPU, FPGA, and the host program match. However, in <Process B-2>, since it is judged based on similarity, there is no guarantee that basic parts such as the number and type of arguments and return values match. Libraries and IP cores are existing know-how, and even if the number and type of arguments and return values do not match, they cannot be changed frequently. For this reason, the user who requests offloading is asked whether the number and type of arguments and return values of the original code will be changed to match the library or IP core. After confirmation and approval, an offload performance test is attempted.

Regarding type differences, if casting between float and double etc. is all that is required, a casting process can be added when creating the processing pattern, and performance measurement trials can begin without user confirmation. Also, if the number of arguments or return values differs between the original program and the library or IP core, for example, arguments 1 and 2 are required in the CPU program and argument 3 is optional, and arguments 1 and 2 are required in the library or IP core, there is no problem with omitting optional argument 3. In such cases, optional arguments may be automatically treated as absent when creating the processing pattern, without user confirmation. Note that if the number and types of arguments and return values match exactly, processing similar to <Process C-1> may be used.

[Function block offloading flowchart]
Next, an outline of the operation of the offload server 200 for offloading a functional block will be described with reference to FIGS.

Flowchart of <Processing A-1>, <Processing B-1>, and <Processing C-1> Figure 7 is a flowchart when the control unit (automatic offload function unit) 210 of the offload server 200 executes <Processing A-1>, <Processing B-1>, and <Processing C-1> in the offload processing of <<Function block offload: common>>.

In step S200, the control unit 210 (see Figure 6) receives the application code and operational conditions.

In step S201, the application code analysis unit 112 (see FIG. 6) analyzes the source code of the application program to be offloaded. Specifically, the application code analysis unit 112 uses a syntax analysis tool such as Clang to analyze the source code, including the library calls and functional processing contained in the code, along with loop statement structures, etc.

In step S202, the replacement function detection unit 213 (see Figure 6) detects an external library call of the application program.

In step S203, the replacement function detection unit 213 obtains replaceable GPU libraries and FPGA IP cores using the library names as keys from the code pattern DB 230. Specifically, the replacement function detection unit 213 compares the identified external library calls with the code pattern DB 230 to obtain the detected replaceable GPU libraries and IP cores as offloadable functional blocks that can be offloaded to the GPU and FPGA.

In step S204, the replacement processing unit 214 replaces the source processing description of the application source code with the destination processing description of the GPU library and FPGA IP core.

In step S205, the replacement processing unit 214 offloads the processing descriptions of the replaced GPU library and FPGA IP core to the GPU and FPGA as functional blocks to be offloaded.

In step S206, the replacement processing unit 214 describes interface processing for calling the GPU library and the FPGA IP core.
In step S207, the execution file creation unit 119 compiles or interprets the created pattern.

In step S208, the performance measurement unit 118 measures the performance of the created pattern in a verification environment ("first performance measurement").
In step S209, the execution file creation unit 119 creates a combination pattern for the pattern that was able to increase the speed during the first measurement.

In step S210, the execution file creation unit 119 compiles or interprets the created combination pattern.
In step S211, the performance measurement unit 118 measures the performance of the created combination pattern in a verification environment ("second performance measurement").

In step S212, the production environment deployment unit 120 selects the pattern with the best performance between the first and second measurements and ends the processing of this flow.

8 is a flowchart of the case where the control unit (automatic offload function unit) 210 of the offload server 200 executes <processing A-2>, <processing B-2>, and <processing C-2> in the offload processing of the functional block. Note that the processing from <processing A-2> may be performed in parallel with the processing from <processing A-1>.

In step S300, the control unit 210 (see Figure 6) receives the application code and operational conditions.

In step S301, the application code analysis unit 112 (see FIG. 6) analyzes the source code of the application to be offloaded. Specifically, the application code analysis unit 112 uses a syntax analysis tool such as Clang to analyze the source code, including the library calls and functional processing contained in the code, along with loop statement structures, etc.

In step S302, the replacement function detection unit 213 (see Figure 6) detects definition description code of a class or structure from the source code.

In step S303, the replacement function detection unit 213 uses a similarity detection tool to obtain replaceable GPU libraries and FPGA IP cores from the code pattern DB 230 using the definition description code of a class or structure as a key.

In step S304, the replacement processing unit 214 replaces the source processing description of the application source code with the destination processing description of the GPU library and FPGA IP core.

In step S305, the replacement processing unit 214 prompts the user if the number or type of arguments or return values differ between the source and destination.

In step S306, the replacement function detection unit 213 offloads the processing descriptions of the replaced GPU library and FPGA IP core to the GPU and FPGA as functional blocks to be offloaded.

In step S307, the replacement processing unit 214 describes interface processing for calling the GPU library and FPGA IP core.

In step S308, the executable file creation unit 119 compiles or interprets the created pattern.
In step S309, the performance measurement unit 118 measures the performance of the created pattern in a verification environment ("first performance measurement").

In step S310, the executable file creation unit 119 creates a combination pattern for the pattern that was able to be sped up during the first measurement.

In step S311, the execution file creation unit 119 compiles or interprets the created combination pattern.
In step S312, the performance measurement unit 118 measures the performance of the created combination pattern in a verification environment ("second performance measurement").

In step S313, the production environment deployment unit 120 selects the pattern with the best performance between the first and second measurements and ends the processing of this flow.

After the GPU and FPGA offload attempt shown in Figures 7 and 8 is completed, the resource ratio and resource amount setting flowchart shown in Figure 5 is executed.

[Hardware configuration]
The offload server according to the first and second embodiments is realized by a computer 900, which is a physical device having a configuration as shown in FIG.
9 is a hardware configuration diagram showing an example of a computer that realizes the functions of the offload servers 1 and 200. The computer 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM 903, a HDD (Hard Disk Drive) 904, an input/output I/F (Interface) 905, a communication I/F 906, and a media I/F 907.

The CPU 901 operates based on a program stored in the ROM 902 or the HDD 904, and controls each processing unit of the offload server 1, 200 shown in Figures 1 and 6. The ROM 902 stores a boot program executed by the CPU 901 when the computer 900 is started, programs related to the hardware of the computer 900, etc.

The CPU 901 controls an input device 910 such as a mouse or keyboard, and an output device 911 such as a display, via the input/output I/F 905. The CPU 901 acquires data from the input device 910 via the input/output I/F 905, and outputs generated data to the output device 911.

The HDD 904 stores programs executed by the CPU 901 and data used by the programs. The communication I/F 906 receives data from other devices via a communication network (e.g., NW (Network) 920) and outputs the data to the CPU 901, and also transmits data generated by the CPU 901 to other devices via the communication network.

The media I/F 907 reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads a program related to the target processing from the recording medium 912 onto the RAM 903 via the media I/F 907, and executes the loaded program. The recording medium 912 is an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disc), a magneto-optical recording medium such as an MO (Magneto Optical disc), a magnetic recording medium, a conductive memory tape medium, or a semiconductor memory, etc.

For example, when the computer 900 functions as the offload server 1, 200 according to the first and second embodiments, the CPU 901 of the computer 900 realizes the functions of the offload server 1, 200 by executing a program loaded onto the RAM 903. In addition, the data in the RAM 903 is stored in the HDD 904. The CPU 901 reads and executes a program related to the target processing from the recording medium 912. In addition, the CPU 901 may read a program related to the target processing from another device via a communication network (NW 920).

[effect]
The effects of the offload server and the like according to the present invention will be described below.
The offload server 1 (see FIG. 1 ) according to the first embodiment is an offload server that offloads specific processing of an application program from a CPU to an offload device, and includes an application code analysis unit 112 that analyzes the source code of the application program, a data transfer specification unit 113 that analyzes the reference relationships of variables used in loop statements of the application program, and for data that may be transferred outside the loop, specifies data transfer using an explicit specification line that explicitly specifies data transfer outside the loop, a parallel processing specification unit 114 that identifies loop statements of the application program and compiles each identified loop statement by specifying a parallel processing specification statement in the offload device, and a parallel processing pattern creation unit 117 that creates a parallel processing pattern that excludes repetitive statements that cause compilation errors from being offloaded, and specifies whether or not to perform parallel processing for loop statements that do not cause compilation errors. The system also includes a performance measurement unit 118 that compiles an application program of a parallel processing pattern, places it on the verification machine 14, and executes processing for measuring performance when offloaded to the offload device, a resource ratio determination unit 115 that determines the ratio of processing times between the CPU and the offload device as a resource ratio based on the performance measurement results, and a resource amount setting unit 116 that sets the resource amounts of the CPU and the offload device based on the determined resource ratio so as to satisfy specified cost conditions.

In this way, appropriate resource amounts for the CPU and offload device can be set when automatically offloading loop statements. This allows server resource amounts to be set efficiently, reducing equipment costs.

The offload server 200 (see FIG. 6) according to the second embodiment is an offload server that offloads specific processing of an application program from a CPU to an offload device including a GPU or PLD, and includes a code pattern DB 230 (storage unit) that stores libraries and IP cores that can be offloaded to the GPU or PLD, an application code analysis unit 112 that analyzes the source code of the application program and detects external library calls contained in the source code, a replacement function detection unit 213 that acquires libraries and IP cores from the code pattern DB 230 using the detected external library calls as a key, and a replacement function detection unit 213 that replaces the processing description of the source of replacement in the source code of the application program with the processing description of the destination library and IP core acquired by the replacement function detection unit 213, and detects the processing description of the replaced library and IP core as the offloaded processing description. The target functional blocks include a substitution processing unit 214 that offloads to a GPU or a PLD, an offload pattern creation unit 215 that creates an interface with a host program and extracts an offload pattern that will be faster by trying whether to offload through performance measurement in a verification environment, an executable file creation unit 119 that compiles the application of the created GPU or PLD processing pattern to create an executable file, a performance measurement unit 118 that places the created executable file in an accelerator verification device and executes processing for performance measurement when offloaded to the GPU or the PLD, a resource ratio determination unit 115 that determines the ratio of processing times between the CPU and the offload device as a resource ratio based on the performance measurement results, and a resource amount setting unit 116 that sets the resource amounts of the CPU and the offload device so as to satisfy a specified cost condition based on the determined resource ratio.

In this way, appropriate resource amounts for the CPU and offload device can be set for automatic offloading of functional blocks. This allows server resource amounts to be set efficiently, reducing equipment costs.

In the offload server 1,200 according to the first and second embodiments, the resource ratio determination unit 115 is characterized in that it determines the resource ratio so that the processing times of the CPU and the offload device are of the same order.

In this way, tasks can be normalized by aligning the processing times of the CPU and offload device to the same order, and the amount of resources can be set appropriately even in a mixed environment where the CPU and accelerator are GPUs, FPGAs, many-core CPUs, etc. In addition, by setting the amount of resources appropriately, slowdowns can be prevented and applications can be run with high cost performance.

In the offload server 1, 200 according to the first and second embodiments, the resource ratio determination unit 115 is characterized in that, when the difference in processing time between the CPU and the offload device is equal to or greater than a predetermined threshold, the resource ratio is set to a predetermined upper limit value.

By doing this, we can prevent a significant increase in the number of VMs by setting an upper limit on the resource ratio.

In the offload server 1, 200 according to the first and second embodiments, the resource amount setting unit 116 is characterized in that it maintains the determined resource ratio and sets the maximum resource amount that satisfies the specified cost conditions.

By doing this, it is possible to set an appropriate amount of resources that meets the user's operational requirements.

In the offload server 1, 200 according to the first and second embodiments, the resource amount setting unit 116 is characterized in that, if the specified cost condition is not satisfied by setting the minimum resource amount while maintaining the determined resource ratio, the resource amount setting unit 116 sets the resource amount so as to start from the minimum for the CPU and offload device and approach the set resource ratio while satisfying the cost condition.

By doing this, it is possible to set an appropriate amount of resources while still satisfying user operational requirements.

The present invention is an offload program for causing a computer to function as the above-mentioned offload server.

By doing this, each function of the offload server 1 can be realized using a general-purpose computer.

Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.
In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

1,200 Offload server 11,210 Control unit 12 Input/output unit 13,130 Memory unit 14 Verification machine (accelerator verification device)
15 Commercial environment 111 Application code specification unit 112 Application code analysis unit 113 Data transfer specification unit 114 Parallel processing specification unit 114a Offload range extraction unit 114b Intermediate language file output unit 115 Resource ratio determination unit 116 Resource amount setting unit 117 Parallel processing pattern creation unit 118 Performance measurement unit 118a Binary file placement unit 119 Executable file creation unit 120 Production environment placement unit 121 Performance measurement test extraction and execution unit 122 User provision unit 125 Application code 131 Test case DB
132 Facility resource DB
133 Intermediate language file 151 Various devices 152 Device having CPU-GPU 153 Device having CPU-FPGA 154 Device having CPU 213 Replacement function detection unit 214 Replacement processing unit 215 Offload pattern creation unit 230 Code pattern DB

Claims (7)

An offload server that offloads a specific process of an application program from a CPU to an offload device,
an application code analysis unit that analyzes a source code of the application program;
a data transfer specification unit that analyzes reference relationships between variables used in a loop statement of the application program, and specifies data transfer using an explicit specification line that explicitly specifies data transfer outside the loop for data that may be transferred outside the loop;
a parallel processing specification unit that specifies loop statements of the application program, and specifies a parallel processing specification statement in the offload device for each of the specified loop statements to compile the loop statements;
a parallel processing pattern creation unit that creates a parallel processing pattern that excludes a loop statement that generates a compilation error from being subject to offloading and specifies whether or not a loop statement that does not generate a compilation error should be processed in parallel;
a performance measurement unit that compiles the application program of the parallel processing pattern, places it in an accelerator verification device, and executes a process for measuring performance when the application program is offloaded to the offload device;
a resource ratio determination unit that determines a ratio of processing times of the CPU and the offload device as a resource ratio based on a performance measurement result;
a resource amount setting unit that sets resource amounts of the CPU and the offload device based on the determined resource ratio so as to satisfy a predetermined cost condition ;
The resource amount setting unit,
If the predetermined cost condition is not satisfied by setting the minimum resource amount while maintaining the determined resource ratio, the resource amount of the CPU and the offload device is set so as to approach the set resource ratio starting from the minimum while satisfying the cost condition.
13. An offload server comprising: An offload server that offloads specific processing of an application program from a CPU to an offload device including a GPU (Graphics Processing Unit) or a PLD (Programmable Logic Device),
A storage unit that stores libraries and IP cores that can be offloaded to the GPU or the PLD;
an application code analysis unit that analyzes a source code of the application program and detects an external library call included in the source code;
a replacement function detection unit that acquires the library and the IP core from the storage unit using the detected external library call as a key;
a replacement processing unit that replaces a processing description of a source of replacement in the source code of the application program with a processing description of a replacement destination of the library and the IP core acquired by the replacement function detection unit , and offloads the replaced processing descriptions of the library and the IP core to the GPU or the PLD as functional blocks to be offloaded;
an offload pattern creation unit that creates an interface with a host program, and extracts an offload pattern that will be faster by testing whether or not to offload through performance measurement in a verification environment;
an execution file creation unit that compiles the application program of the created GPU or PLD processing pattern to create an execution file;
a performance measurement unit that places the created executable file in an accelerator verification device and executes a process for measuring performance when the executable file is offloaded to the GPU or the PLD;
a resource ratio determination unit that determines a ratio of processing times of the CPU and the offload device as a resource ratio based on a performance measurement result;
a resource amount setting unit that sets resource amounts of the CPU and the offload device based on the determined resource ratio so as to satisfy a predetermined cost condition ;
The resource amount setting unit,
If the predetermined cost condition is not satisfied by setting the minimum resource amount while maintaining the determined resource ratio, the resource amount of the CPU and the offload device is set so as to approach the set resource ratio starting from the minimum while satisfying the cost condition.
13. An offload server comprising: The resource ratio determination unit is
3. The offload server according to claim 1, wherein the resource ratio is determined so that the processing times of the CPU and the offload device are of the same order. The resource ratio determination unit is
3. The offload server according to claim 1, wherein when a difference between a processing time of the CPU and the offload device is equal to or greater than a predetermined threshold, the resource ratio is set to a predetermined upper limit value. The resource amount setting unit,
3. The offload server according to claim 1, further comprising: setting a maximum resource amount that satisfies the predetermined cost condition by maintaining the determined resource ratio. An offload control method for an offload server that offloads a specific process of an application program from a CPU to an offload device, comprising:
The offload server includes:
analyzing the source code of the application program;
a step of analyzing reference relationships of variables used in a loop statement of the application program, and for data that may be transferred outside the loop, specifying data transfer using an explicit specification line that explicitly specifies data transfer outside the loop;
identifying loop statements of the application program, and for each of the identified loop statements, specifying a parallel processing specification statement for the offload device and compiling the loop statements;
creating a parallel processing pattern for specifying that a loop statement that generates a compilation error is not to be offloaded and that a loop statement that does not generate a compilation error is to be processed in parallel or not;
A step of compiling the application program of the parallel processing pattern, disposing the application program in an accelerator verification device, and executing a process for measuring performance when the application program is offloaded to the offload device;
determining a ratio of processing times of the CPU and the offload device as a resource ratio based on a performance measurement result;
setting the resource amounts of the CPU and the offload device so as to satisfy a specified cost condition based on the determined resource ratio, and if the specified cost condition is not satisfied by setting a minimum resource amount that maintains the determined resource ratio, setting the resource amounts of the CPU and the offload device so as to approach the set resource ratio starting from a minimum while satisfying the cost condition . An offload program for causing a computer to function as the offload server according to any one of claims 1 to 5 .

Images