This is an old revision of the document!


PLOMO2: A new generation of tools to navigate and profile programs

An INRIA Equipe Associee between RMOD and PLEIAD, a continuation of the PLOMO Equipe Associee named after cerro El Plomo, the highest peak (5424 meters) that is visible from Santiago. Foto by by Thiago "James".

Introduction

Performing effective software development and maintenance are best achieved with effective tool support. Provided by a variety of tools, each one presenting a specific kind of information supporting the task at hand. The goal of the first PLOMO was to develop new meta tools to improve and bring synergy in the existing infrastructure of Pharo (for software development) and the Moose software analysis platform (for maintenance). With Plomo2, we want to build on top of this work and invent a new generation tools to navigate and profile programs.

The hypotheses that Plomo2 will seek to verify are:

  • Use of reflection enables new profiling techniques
  • Use of visualization in a programming environment improves programmer performance

The overall objectives of Plomo2 are:

  1. Infrastructure for profiling programs and recording programmer activity.
  2. Visual software maps defined in a flexible and agile fashion
  3. Combining dynamic information with visualization to improve the development environment
  4. Empirical evaluation of this environment

All the efforts will be performed on Pharo and Moose, two platforms heavily used by the RMoD and Pleiad teams.

Work plan

Work Package One:

For both for program profiling and programmer activity recording, we need a way to deeply hook into the system. We plan for this to improve the reflective infrastructure of Pharo.

During the previous period we worked on OPAL, a new compiler toolchain for Pharo. In essence the work on the compiler framework does not move forward the state of the art in compiler technology. However it has already been proven to be a crucial building block. It provided the fundamentals for many research experiments and new features for Pharo. For example, Pleiad and RMoD collaborated on a novel type system for Pharo that was realized using the new compiler infrastructure. Other examples are the many new features of Pharo 3 that use parts of the new compiler chain, like AST-based navigation in the editor or breakpoints.

The Opal flexible compiler infrastructure will allow us to build a new generation of reflective infrastructure. Such infrastructure is keys to support tools (profilers) and new language design (AOP, proxies, isolation). We are now in a position to perform a new iteration on the reflective layer and metaobject protocol for Pharo. In 2005 we started to develop new reflective foundations and we learned a lot (partial reflection, metalevel recursion handling, object-centric on the fly propagation, proxies, isolation). We are now in a situation to step back and redesign the reflective layer based on all the cases we identified and developed.

The results of this work package will be used and thus validated by WP3. In addition, we plan to realize the next generation of the AOP framework Phantom using the reflection framework. This especially makes sure that the framework is general and not just limited for the use in tools.

Work Package Two:

Plomo made a lot of progress by realizing the Roassal visualisation framework. We plan to enhance the infrastructure to visually render software-related data. We will investigate two complementary directions:

  1. Interaction and scalability in Roassal 2D. Roassal is an Agile Visualization Engine that offers large range of options to make the user interact with the visualization. Visualization software require particular customizations to cope with large amount of data. Navigation should support interactive and semantic zooming, which is an essential feature for navigating large amount of data elements. Additional layers, including drawing dependency matrices, vectorial and interactive charts are required
  2. Visual representation of 3D environment gives a sensation of immersion that is often key to identify some patterns in a data structure. Current programming environments exploiting 3D are either fixed to a particular representation or unfit to be used as the base of a programming environment

Work Package Three:

The goal of the third work package is to build on top of both WP1 and WP2: Combining dynamic information with visualization to improve the development environment.

  1. Visualization to navigate and browse source code – System complexity is a polymetric view efficient at representing class hierarchies in a software source code. It has been shown that system complexity is effective for software maintenance. We will augment the range of application of it by using it to navigate and browser application source code.
  2. Visualizing dynamic information to navigate source code. The static view of code should be enhanced with dynamic information gathered from program execution. A good example here is visualization of profiling and tracing information.
  3. Context based on history – It has been shown that analyzing past user actions may precisely infer what will be the next user actions. Unfortunately, history is poorly supported in traditional programming environment. Making history a first class element in the programming environment is the topic of this work package.

This work package will therefore validate both WP1 and WP2. We will validate the ideas developed as part of WP3 by carrying out an empirical evaluation.

Achievements

Scientific Results 2014

WP 1

Progress on a new iteration on the reflective layer and metaobject protocol for Pharo: We have implemented the Slots Mechanism in Pharo, which resulted in a first-class reification of instance variables and their accessing through assignment and reading. This allows us to hook into reading and writing of instance variables and perform meta-level operations on them, e.g. adding breakpoints to the reading of a specific instance var, or a transparent notification mechanism for when a specific instance variable is written. The latter can be used, e.g. in WP3 to dynamically update a visualization of the object that contains the instance variable whenever the variable changes.

WP 2

Interaction and scalability in Roassal 2D. We have improved the FFI support of Pharo to use external libraries. With that, we created binding to SDL2 which allowed to move the window management code from the Virtual Machine to Pharo (OSWindow).

With the new FFI and the OSWindow functionality, bindings to OpenGL has been created. The Woden 3D graphics engine is a first demo that uses this foundation to bring the ideas of Roassal to 3D. Video.

WP 3

Live Robot Programming (LRP) as a case for the first two points of WP3: Visualization to navigate and browse source code, Visualizing dynamic information to navigate source code. Firstly, in LRP the (Domain-Specific) source code is shown in a (Domain-Specific) visualization made in Roassal2 and Spec. Clicking on the entities in the visual representation causes a navigation action in the source code of the program. Secondly, in the LRP visualization the dynamic information of the state of variables in scope is shown, as well as which piece of the code is currently running, and this is partially enabled by Slots. Future work of LRP will improve the dynamic visualization, adding a log of key information that allows to replay the visualization from (and to) a given point, which should be possible by using Slots.

Scientific Results 2015

WP 1

Expressive Reflective Infrastructure: We worked extensively on the new reflection infrastructure of Pharo as well as First Class Variables. Reflective operations in the form of meta-links can be added to any expression tree. All variables can now be annotated with meta data. Reflectivity has been improved to allow Meta-Links to be put on First Class Variables. All work has been integrated in the current Pharo5 development version and is used already by other projects of RMOD and a research project of the Universit of Lugano/Switzerland. The stay of Marcus Denker in Chile in November has the goal to make the new reflective infratructure to be reliable enough for other PLOMO sub projects to use (namely LRP and Profilers).

WP 2

Advanced Visualization Techniques: Roassal 2D continues to improve. The 2D infrastructure work (OSWindow) has been taken up by another Project (Inria in collaboration with Thales). It has been extended with multi-touch support and a gesture engine. Besides pre-defined low level gestures (e.g. pinch or multi-finger swipe), the engine is extensible with user-defined gestures. In addition, work on Roassal 3D is progressing. Roassal 3D is now based on the Woden 3D engine. We hope to see a first release next year.

WP 3

Analyzing Dynamic Information: The work on Live Robot Programming (LRP) continues. The visit of Marcus Denker to Chile in spring 2015 helped to make serious progress on the base implementation infrastructure: The LRP protoype now is based in the First Class Variables (Slots and Globals) that are available in Pharo4.

Publications

  • Johan Fabry, Andy Kellens, Simon Denier, Stephane Ducasse – AspectMaps: Extending Moose to Visualize AOP Software – Science of Computer Programming, volume 79, number 1, pp.6-22, January 2014, Elsevier
  • Johan Fabry, Romain Robbes, Marcus Denker – DIE: A Domain Specific Aspect Language for IDE Events – Journal of Universal Computer Science, volume 20, number 2, pp.135-168, February 2014
  • Esteban Allende, Oscar Callaú, Johan Fabry, Éric Tanter, Marcus Denker – Gradual Typing for Smalltalk – Science of Computer Programming, volume 96, number 1, pp.52-69, December 2014, Elsevier
  • Benjamin Van Ryseghem, Stéphane Ducasse, Johan Fabry – Seamless Composition and Reuse of Customizable User Interfaces with Spec – Science of Computer Programming, volume 96, number 1, pp.34-51, December 2014, Elsevier
  • Alexandre Bergel, Sergio Maass, Stéphane Ducasse, Tudor Girba – A Domain-Specific Language For Visualizing Software Dependencies as a Graph. Proceedings of 2nd IEEE Working Conference on Software Visualization (VISSOFT NIER) pdf
  • A book chapter about the Spec user interface description framework has been finished for a new Pharo Book. pdf
  • LinuxMag articles on Roassal and the Charter charting framwork.
  • Leonardo Humberto Silva, Marco Tulio Valente, Nicolas Anquetil, Daniel Carlos Hovadick Félix, Alexandre Bergel, Anne Etien – JSClassFinder: A Tool to Detect Class-like Structures in JavaScript. Proceedings of Brazilian Conference on Software: Theory and Practice (CBSOFT'15)
  • Miguel Campusano and Johan Fabry: “From Robots to Humans: Visualizations for Robot Sensor Data.” In the third IEEE Working Conference on Software Visualization (VISSOFT 2015), Sep 27-28, 2015 , IEEE press
  • Damien Cassou, Stéphane Ducasse, Luc Fabresse, Johan Fabry, Sven Van Caekenberghe: Enterprise Pharo. This is the third volume of a series of books covering Pharo and its community. http://files.pharo.org/books/enterprisepharo/

Publications in preparation

  • Book Agile Visualization: http://agilevisualization.com. The work on a book on Agile Visualization with Pharo and Roassal has been started.
  • Miguel Campusano and Johan Fabry: “ Live Robot Programming: The Language, its Implementation, and Robot API Independence” Submitted to Elsevier journal Science of Computer Programming, September, 2015
  • “Identifying Classes in Legacy JavaScript Source Code” by Leonardo Humberto Silva, Marco Tulio Valente, Alexandre Bergel,Nicolas Anquetil, and Anne Etien. To be submitted to TSE.

Software artifacts

  • Pharo: Pharo3 has been released Spring 2014, Pharo4 has seen active development with a release planned for early 2015. http://pharo.org
  • Pharo: Pharo4 has been released Spring 2015, Pharo5 has seen active development with a release planned for early 2016. http://pharo.org
  • Moose: Version 5.0 was relseased. http://moosetechnology.org
  • Roassal2D: Fully based on Athens, the vector graphics support in Pharo
  • Roassal3D: first version of a 3D version of Roassal. https://vimeo.com/141916068
  • Woden: a new 3D graphic engine, the base of the next generation of Roassal3D.
  • Improvement of the FFI description pdf
  • Reflectivity has been integrated in Pharo5
  • OSWindow has been integrated in Pharo4.
  • The Opal Compiler has been further improved. It has been extended with support for Slots.

Event organization

  • International Workshop on Smalltalk Technologies (IWST), co-located at ESUG 2014. Paper awards were sponsored by Lam Research.
  • International Workshop on Smalltalk Technologies (IWST), co-located at ESUG 2015. Paper awards were sponsored by Lam Research.
  • International Workshop on Smalltalk Technologies (IWST), co-located at ESUG 2016. Paper awards were sponsored by Lam Research.

Visits

2014 Santiago To Lille:

  • Ronie Saldago: 24/08/2014 until 07/09/2014. Subject was FFI and OSWindow.
  • Miguel Campusano: 16/08/2014 until 11/09/2014. Subject was Slots and visual representation of code.
  • Alexandre Bergel: 13/12/2014 until 01/01/2015. Subject: system support for advanced profiling.
  • Juraj Kubelka: 06/12/2014 until 19/12/2014. First visit to RMoD to plan future collaboration.
  • Johan Fabry (January 2015 for Pharo Days Lille, PLEIAD funded)
  • Alexandre Bergel (01/07/2015 until 27/07/2015)
  • Johan Fabry (July 2015 for ESUG, PLEIAD funded)
  • Pierre Chanson (July 2015 and September 2015)
  • Miguel Campusano (20/09/2015-07/10/2015)
  • Alexandre Bergel (Dec 2015)

Lille to Santiago:

  • Marcus Denker: 2/1/2015 until 20/1/2015, sponsored by PLEIAD. Subject is Slots.
  • Marcus Denker: Nov 2015 payed by PLEIAD. Subject is Reflection Framework.