Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
Last revisionBoth sides next revision
research:software:gradualtalk:ggt [2014/03/24 16:35] – created eallenderesearch:software:gradualtalk:ggt [2014/03/31 12:26] – eallende
Line 2: Line 2:
  
 The benchmarks used in the paper "Gradual Gradual Typing" can be downloaded here: The benchmarks used in the paper "Gradual Gradual Typing" can be downloaded here:
-{{research:software:gradualtalk:ggt-benchmarks.tar.gz|OOPSLA paper benchmark suite}}.+{{research:software:gradualtalk:ggt-benchmarks.zip|OOPSLA paper benchmark suite}}. 
 +This file also include example images for SGGT and RGGT.
  
-For instructions on how to use these benchmarks, please read the included ''readme.txt''. 
  
-**Still need to generate tar of the benchmark suit**+For instructions on how to use these benchmarks and examples, please continue reading this page or read the included ''readme.txt''.
  
-The benchmark suite is ready to run in a Windows machine. To run in another OS, the corresponding [[http://files.pharo.org/vm/cog/|Cog VM]] needs to be downloaded. The benchmarks have only been run on Linux and MacOS. For MacOS, the path to the actual virtual machine inside the application package is ''CogVM.app/Contents/MacOS/CogVM'' Also on Mac, the VM does not run completely headless, please ignore the window that pops up when the benchmarks are run.+The benchmark suite is ready to run in a Windows or Linux machine. To run in another OS with bash, the corresponding [[http://files.pharo.org/vm/cog/|Cog VM]] needs to be downloaded. The benchmarks have only been run on Windows and Linux. For MacOS, the path to the actual virtual machine inside the application package is ''CogVM.app/Contents/MacOS/CogVM'' Also on Mac, the VM does not run completely headless, please ignore the window that pops up when the benchmarks are run. 
 + 
 +===== Contents of the image =====  
 + 
 + 
 +This image includes: 
 +  * examples of SGGT and RGGT 
 +  * the micro and macro benchmarks 
 + 
 +Also, the Windows Cog VM is included in the image. To run in an OS with bash, 
 +put the corresponding Cog VM in the "vm" directory and modify the $VM variable of the linux 
 +benchmarks scripts. 
 + 
 +===== Examples =====  
 +The example images of SGGT and RGGT are located in the example folder. 
 + 
 +To run the SGGT image, execute: 
 + 
 +  Win:   .\sggt.bat 
 +  Linux: ./sggt.sh 
 + 
 +To run the RGGT image, execute: 
 + 
 +  Win:   .\rggt.bat 
 +  Linux: ./rggt.sh 
 + 
 +==== Qualifier syntax ==== 
 +The syntax for the qualifiers are: 
 + 
 +  Up qualifier:   ^^   Ex: ^^Integer, ^^(->String) 
 +  Down qualifier: vv   Ex: vvString, vv(Integer->vvString) 
 + 
 +===== Benchmarks =====  
 +The micro and macro benchmark are configured to run in a machine with Windows or Linux. 
 + 
 +==== Micro-benchmarks ====  
 + 
 +To run the micro-benchmarks, go to the directory "benchmark/micro" and execute: 
 + 
 +  Win:   .\bench.bat 
 +  Linux: ./bench.sh 
 + 
 + 
 +The results of the microbenchmarks are stored in the results directory inside the directory of the 
 +microbenchmarks ("benchmark/micro/results") in the file results-X.txt, where X is the semantics used: 
 +  - GT for non-gradual Gradual Typing 
 +  - RGGT for Relaxed Gradual Gradual Typing 
 + 
 +In the results-X.txt file, each line states: 
 +  - the size of the collection. 
 +  - the time taken in:  
 +    - Wrapper Creation  , NQ, Wrapped. 
 +    - Closure Evaluation, NQ, Wrapped. 
 +    - Wrapper Creation  , NQ, Unwrapped. 
 +    - Closure Evaluation, NQ, Unwrapped. 
 +    - Wrapper Creation  , NQ, NoDyn. 
 +    - Closure Evaluation, NQ, NoDyn. 
 +    - Wrapper Creation  , DQ, Unwrapped (only RGGT). 
 +    - Closure Evaluation, DQ, Unwrapped (only RGGT). 
 +    - Wrapper Creation  , DQ, NoDyn (only RGGT). 
 +    - Closure Evaluation, DQ, NoDyn (only RGGT). 
 +    - Wrapper Creation  , UQ, Unwrapped (only RGGT). 
 +    - Closure Evaluation, UQ, Unwrapped (only RGGT). 
 +    - Wrapper Creation  , UQ, NoDyn (only RGGT). 
 +    - Closure Evaluation, UQ, NoDyn (only RGGT). 
 +   
 +Legend: NQ=No Qualifiers, DQ=Down arrow qualifier, UQ=Up arrow qualifier 
 + 
 +==== Macro-benchmarks ====  
 + 
 +To run the macro-benchmarks, go to the directory "benchmark/macro" and execute: 
 + 
 +Win:   .\bench.bat 
 +Linux: ./bench.sh 
 + 
 +The results of the microbenchmarks are stored in the results directory inside the directory of the 
 +macrobenchmarks ("benchmark/macro/results") in the file results-X.txt, where X is either:  
 +  -GT-Wrapped 
 +  -GT-Unwrapped 
 +  -RGGT-Wrapped 
 +  -RGGT-Wrapped-NQ 
 +  -RGGT-Wrapped-DQ 
 +  -RGGT-Wrapped-UQ 
 +   
 +Legend: NQ=No Qualifiers, DQ=Down arrow qualifier, UQ=Up arrow qualifier 
 + 
 +Each file containts the time taken to run the macro-benchmark in that variant.