Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
research:software:gradualtalk:ggt [2014/03/28 07:30] – eallenderesearch:software:gradualtalk:ggt [2014/06/09 15:06] (current) – jfabry
Line 1: Line 1:
-====== Gradual Gradual Typing: Benchmarks ======+====== Confined Gradual Typing: Benchmarks ======
  
-The benchmarks used in the paper "Gradual Gradual Typing" can be downloaded here: +You probably want to be here: [[CGT]]
-{{research:software:gradualtalk:ggt-benchmarks.zip|OOPSLA paper benchmark suite}}. +
-This file also include example images for SGGT and RGGT. +
- +
- +
-For instructions on how to use these benchmarks and examples, please continue reading this page or read the included ''readme.txt''. +
- +
-The benchmark suite is ready to run in a Windows or Linux machine. To run in another OS with bash, the corresponding [[http://files.pharo.org/vm/cog/|Cog VM]] needs to be downloaded. The benchmarks have only been run on Windows and Linux. For MacOS, the path to the actual virtual machine inside the application package is ''CogVM.app/Contents/MacOS/CogVM'' Also on Mac, the VM does not run completely headless, please ignore the window that pops up when the benchmarks are run. +
- +
- +
-===== Contents of the image =====  +
- +
- +
-This image includes: +
-  * examples of SGGT and RGGT +
-  * the micro and macro benchmarks +
- +
-Also, the Windows Cog VM is included in the image. To run in an OS with bash, +
-put the corresponding Cog VM in the "vm" directory and modify the $VM variable of the linux +
-benchmarks scripts. +
- +
-===== Examples =====  +
-The example images of SGGT and RGGT are located in the example folder. +
- +
-To run the SGGT image, execute: +
- +
-  Win:   .\sggt.bat +
-  Linux: ./sggt.sh +
- +
-To run the RGGT image, execute: +
- +
-  Win:   .\rggt.bat +
-  Linux: ./rggt.sh +
- +
- +
-===== Benchmarks =====  +
-The micro and macro benchmark are configured to run in a machine with Windows or Linux. +
- +
-==== Micro-benchmarks ====  +
- +
-To run the micro-benchmarks, go to the directory "benchmark/micro" and execute: +
- +
-  Win:   .\bench.bat +
-  Linux: ./bench.sh +
- +
- +
-The results of the microbenchmarks are stored in the results directory inside the directory of the +
-microbenchmarks ("benchmark/micro/results") in the file results-X.txt, where X is the semantics used: +
-  - GT for non-gradual Gradual Typing +
-  - RGGT for Relaxed Gradual Gradual Typing +
- +
-In the results-X.txt file, each line states: +
-  - the size of the collection. +
-  - the time taken in:  +
-    - Wrapper Creation  , NQ, Wrapped. +
-    - Closure Evaluation, NQ, Wrapped. +
-    - Wrapper Creation  , NQ, Unwrapped. +
-    - Closure Evaluation, NQ, Unwrapped. +
-    - Wrapper Creation  , NQ, NoDyn. +
-    - Closure Evaluation, NQ, NoDyn. +
-    - Wrapper Creation  , DQ, Unwrapped (only RGGT). +
-    - Closure Evaluation, DQ, Unwrapped (only RGGT). +
-    - Wrapper Creation  , DQ, NoDyn (only RGGT). +
-    - Closure Evaluation, DQ, NoDyn (only RGGT). +
-    - Wrapper Creation  , UQ, Unwrapped (only RGGT). +
-    - Closure Evaluation, UQ, Unwrapped (only RGGT). +
-    - Wrapper Creation  , UQ, NoDyn (only RGGT). +
-    - Closure Evaluation, UQ, NoDyn (only RGGT). +
-   +
-Legend: NQ=No Qualifiers, DQ=Down arrow qualifier, UQ=Up arrow qualifier +
- +
-==== Macro-benchmarks ====  +
- +
-To run the macro-benchmarks, go to the directory "benchmark/macro" and execute: +
- +
-Win:   .\bench.bat +
-Linux: ./bench.sh +
- +
-The results of the microbenchmarks are stored in the results directory inside the directory of the +
-macrobenchmarks ("benchmark/macro/results") in the file results-X.txt, where X is either:  +
-  -GT-Wrapped +
-  -GT-Unwrapped +
-  -RGGT-Wrapped +
-  -RGGT-Wrapped-NQ +
-  -RGGT-Wrapped-DQ +
-  -RGGT-Wrapped-UQ +
-   +
-Legend: NQ=No Qualifiers, DQ=Down arrow qualifier, UQ=Up arrow qualifier +
- +
-Each file containts the time taken to run the macro-benchmark in that variant.+