====== Live Robot Programming ====== Live Robot Programming (LRP) is a live programming language designed for the creation of the behavior layer of robots. It is fundamentally a nested state machine language built with robotics applications in mind, but it is not bound to a specific robot middleware, API or OS. LRP is open source, download instructions [[research:software:lrp#Downloads|are below]], [[https://github.com/jfabry/LiveRobotProgramming|example code]] can be found online and there is a public [[https://github.com/jfabry/LiveRobotProgramming/issues|bug tracker]]. We all like videos of robots, right? Here are some of LRP on a few different robots for your viewing pleasure. * [[https://www.youtube.com/watch?v=2bKEEVfuMic|Moving the Duckiebot platform using LRP]] (This uses ROS to control the robot.) * [[https://www.youtube.com/watch?v=fl21wsb_a3c|How does a PR2 move through a door with LRP]]? (This uses ROS to control the robot.) * [[https://www.youtube.com/watch?v=ENmuGU4gBh4|Flying a Parrot AR.Drone 2]], using AR tags. (This uses the Parrot API to control the drone.) * [[https://www.youtube.com/watch?v=q1jGDtaW41U|Making a NAO robot follow a ball]]. (This uses the NAO API to control the robot.) * [[https://www.youtube.com/watch?v=rtSKH-FBbzA|PR2 pick and place interaction with speech control]]. (This uses ROS to control the robot.) **NEWS**: * Miguel presented LRP at the [[https://jpsandoval.github.io/sla2017/|Cochabamba Summer School 2017]], at the Universidad Mayor the San Simón, Cochabamba, Bolivia. * [[http://pharo.org/news/AR.Drone|Parrot AR.Drone 2 support is finished!]] * [[http://pharo.org/news/Nao-robot|A first version of NAO support is out!]] * Journal paper defining LRP, the functioning of the interpreter and bridges to robot API accepted for Science of Computer Programming, a [[http://pleiad.dcc.uchile.cl/papers/2016/campusanoAl-scp16.pdf|preprint is available]]. * NIER Paper on visualizing robotic sensors accepted at [[http://vissoft.dcc.uchile.cl/|VISSOFT 2015]], a [[http://pleiad.dcc.uchile.cl/papers/2015/campusanoFabry-vissoft2015.pdf|preprint is available]]. * Johan presented LRP at the [[http://www.esug.org/wiki/pier/Conferences/2015|ESUG 2015]] conference. A video of the talk is available [[https://youtu.be/0CYMkxV919s?list=PLJ5nSnWzQXi9rnk77bTf-juPt5I6-ered|as part of the conference playlist]] on Youtube. * LRP was part of Johan's talk at the [[http://www.roboticsday.org/|robotics day]] conference, claimed to be the largest robotics event in Latin America. Have a look at one minute of LRP coding: making a state machine for the tick-tock of an oldfashioned analog watch:
As LRP is not bound to a specific robot infrastructure, bridges to a given infrastructure are add-ons to the language. The language can therefore also be used 'simply' as a nested state machine live programming language! Currently LRP provides a bridge to [[http://www.ros.org|ROS]], to the Lego Mindstorms EV3 via [[http://www.phratch.com/jetstorm|JetStorm]] and to the [[http://www.parrot.com/usa/products/ardrone-2/|Parrot AR.Drone 2]]. LRP is implemented in [[http://pharo.org|Pharo Smalltalk]], so basic knowledge of Smalltalk is required to be able to write useful code. But don't panic, Pharo itself comes with a set of tutorials that are sufficient for being able to use LRP. ==== Overview of the Language Features ==== LRP programs describe nested state machines: a state of a machine may contain (several) complete state machine(s), whose states may again contain a machine, and so on. Next to states, machines may also define transitions and events. Events are named actions (= Smalltalk blocks) that specify the conditions for transitions. LRP has four kinds of transitions: * normal transitions: the usual ones * epsilon transitions: who trigger automatically when the state is active * timeout transitions: who trigger after their timeout (in milliseconds) given as a literal or as a variable reference * wildcard transitions: who have no specific source state, they consider all of the states in their machine as the source state. The following is an example state machine (called ''esc'') that models the escapement of a mechanical clock (the part responsible for the tick-tock). It consists of two states (''tick'' and ''tock'') and the timeout transitions between them. To the right is its visual representation in the LRP editor. {{ research:software:ticktock.png}} (machine esc (state tick) (state tock) (ontime 500 tick -> tock) (ontime 500 tock -> tick) ) To start interpreting this machine, a ''spawn'' statement needs to be specified. Spawns can be declared either at the top level, for the top level machine or as the ''onentry'' action of a state (see below for ''onentry''), for nested machines. The video at the top of this page shows the process of writing and running this code. (spawn esc tick) States may also contain actions: an on entry action (''onentry''), an on exit action (''onexit'') and a running action (''running''). Actions are Smalltalk blocks. When a state becomes active, its on entry action is executed once, completely. When the state stops being active, its on exit action is executed once, completely. While the state is active, its running action is executed as the main part of the interpretation loop. In other words, the block is executed repeatedly, and between each execution of the block the events are checked. A machine can also define variables. Global variables may also be defined, outside of the root machine. Variables must be given a value when they are defined, the value is the result of evaluating a Smalltalk block. This block has in scope all variables that are lexically in scope. All actions (i.e. ''onentry'', ''onexit'', ''running'' blocks) may read, send messages to, and set all variables in their lexical scope. For example, to add a seconds counter to the escapement, the code is as below. We show a screenshot of the complete editor here. It shows the code pane (left), the tree of machines and current variables (center), and the visualization of the machine (right). {{ research:software:esc-secs.png }} Another example is a bit more complicated: a resettable timer that shows minutes as a variable, and seconds (rounded to 10) as a state. To the right of the code we show the relevant part of the editor. The ''go'' transition is an epsilon transition, the ''reset'' transition is a wildcard transition because of the ''*'' before the arrow replacing the source state. {{ research:software:timer.png}} ;; a resettable timer with 10 sec intervals (var minute := [0]) (machine timer (state zero) (state ten) (state twenty) (state thirty) (state fourty) (state fifty (onexit [minute := minute + 1])) (ontime 10000 zero -> ten toten) (ontime 10000 ten -> twenty totwenty) (ontime 10000 twenty -> thirty tothirty) (ontime 10000 thirty -> fourty tofourty) (ontime 10000 fourty -> fifty tofifty) (ontime 10000 fifty -> zero tozero) (var doreset := [0]) (state init (onentry [minute := 0. doreset := 0])) (on resetting *-> init reset) (eps init -> zero go) (event resetting [doreset = 1]) ) (spawn timer zero) Below is a video that shows this machine in action, and demonstrates that when the ''doreset'' variable is set to 1 * the wildcard transition occurs to ''init'', * whose onentry action resets ''minute'' to 0 and ''doreset'' to 0, * which is then followed by the epsilon transition to ''zero''.
As a variation on the clock theme, below is the example for a stopwatch. The values of the variables influence whether the stopwatch is running or stopped, and if it should be reset. Also, the stopwatch state machine is an example that uses all transition types available in LRP. ;;; Stopwatch (machine stopwatch (var start := [false]) (var reset := [false]) (var seconds := [0]) (state waiting ) (on starting waiting -> tick) (event starting [start]) (state tick) (state tock (onexit [seconds := seconds + 1])) (ontime 500 tick -> tock ) (ontime 500 tock -> tick ) (on reset *-> resetting) (event reset [reset]) (state resetting (onentry [reset := false. seconds := 0] )) (eps resetting -> tick tet) (on stop *-> waiting tes) (event stop [start not]) ) (spawn stopwatch waiting)
==== Advanced Features ==== === Nested Machines === As LRP is a nested state machine language, a state of a machine may contain (several) complete state machine(s), whose states may again contain a machine, and so on. Machines can be spawned when the state is entered by having the ''onentry'' statement contain a ''spawn'' statement instead of an action block. The machine specified in such a spawn can be any machine that is in lexical scope. This allows the machine to be spawned to be defined in the spawning state or any parent state, recursively up to the root (including other machines defined at root level). Note that if there is **no** ''onentry'' spawn, a machine that is contained lexically in the state is **not** executed. When a state with a nested machine stops being active, the nested machine is stopped and discarded. As part of this process, the on exit action in the active state of this nested machine is performed after any machines nested in that state are stopped and discarded (and so on recursively). A brief (contrived) example is as follows: (machine root (state r1 (machine nest1 (state n1 (machine nest2 (state n1n2)) (onentry (spawn nest2 n1n2)) ) (ontime 500 n1->n2) (state n2 (machine nest3 (state n2n3)) (onentry (spawn nest3 n2n3)) ) (state n3) (ontime 500 n2->n3)) (onentry (spawn nest1 n1)) ) (state r2) (ontime 2000 r1->r2) (ontime 500 r2->r1) ) (spawn root r1) === Concurrency === It is possible to have multiple machines running at the same time, i.e. to have multiple machines each with an active state, without these machines being nested. To achieve this, multiple ''spawn'' statements are specified, either at top level or as ''onentry'' actions of a state. Concurrency obeys the following rules: * If a spawn statement specifies a machine that is already running, i.e. that already has an active state, the spawn fails with an error. * exiting a state that has multiple nested machines concurrently running means exiting all of these machines * removing a top-level spawn does NOT stop the machine that was spawned * adding or removing a spawn of a state means changing the state. So if the state is active (possibly having multiple nested machines running) the program is restarted. The scheduling of machine execution is fair and predictable: * all top level spawned machines perform one interpretation step for each interpreter step. This happens sequentially and the order is the lexical order of the spawn statements. * all nested machines perform one interpretation step for each interpretation step of the state. This happens sequentially and the order is the lexical order of the spawn statements. Note that this implies recursion when an active state of a nested machine has concurrent nested state machines. Put succinctly, interpretation of multiple running machines is equal to a depth-first traversal of the running machines tree. There are two brief (contrived) examples: (machine m (state s) (state t) (ontime 1500 s->t) (ontime 1500 t->s) ) (machine n (state a) (state b) (state c) (ontime 2000 a->b) (ontime 2000 b->c) (ontime 2000 c->a) ) (spawn m s) (spawn n a) (machine m (state s) (state t) (ontime 1500 s->t) (ontime 1500 t->s) ) (machine n (state a) (state b) (state c) (ontime 2000 a->b) (ontime 2000 b->c) (ontime 2000 c->a) ) (machine o (state p (onentry (spawn m s)) (onentry (spawn n a))) ) (spawn o p) === Exit transitions === In a nested machine it is possible to define transitions that go to a state of the parent machine, effectively exiting the nested machine. Such transitions are like normal transactions, except that their keyword is ''exit'' and the destination state should be a state of the parent machine. Note that exit transitions are in fact syntactic sugar. A simple example is as follows. As soon as the ''out'' variable is set to ''true'', the nested machine is exit. (machine root (var out := [false]) (state one (machine nested (state onen) (exit goout onen->two) (event goout [out])) (onentry (spawn nested onen)) ) (state two) ) (spawn root one) === Eventless transitions === It can become tedious for transitions to need an event as a trigger, since it requires the definition of an event as a separate statement. This is especially tedious when the transition is the only that references that event. To ease this tedium, transitions also accept a block instead of an event name. This block should return true for the transition to trigger. Eventless transitions are in fact syntactic sugar: an event is generated and added to the machine, with as action block the block that was specified in the transition, and the transition instead then refers to that event. === User interface: Transition to and Jump to === The LRP user interface allows for the user to force a machine in a given state. By right-clicking on a state in the visualisation a menu appears, with the option to ''transition to'' or ''jump to''. The former acts as if a transition is added from the currently active state to the selected state, and this transition is removed immediately after it is taken. The latter also transitions to the given state, however **without** running the ''onexit'' and ''onentry'' actions of all affected states. (Recall that if a state has a nested machine, its active state ''onexit'' actions are normally also executed.) Transition to and jump to also combine with concurrency (see above for concurrency): if this machine and none of its parents is running, the machine is spawned as a top-level spawn. If it (or its parents) is running, the active state of that running machine is considered as the state from which the transition or jump starts. ==== Downloads ==== LRP is available on SmalltalkHub, under the MIT license. To be able to use it, you first need to download [[http://pharo.org/download|Pharo 5]] and basic knowledge of Pharo is required. Note that Pharo itself comes with the tutorials required to get you up to speed. === Installing LRP === You can download [[https://ci.inria.fr/pharo-contribution/job/LiveRobotProgramming/PHARO=50,VERSION=development,VM=vm/lastSuccessfulBuild/artifact/LiveRobotProgramming.zip|a working image]] of the latest development version from our continuous integration server. This image is ready to work, containing LRP and any other project that LRP needs. In this image you will have a new menu entry in the World menu that opens the LRP UI. (Click on the background to get the World menu.) We have example code online [[https://github.com/jfabry/LiveRobotProgramming/tree/master/|in our GitHub repository]]. To work with [[http://www.ros.org|ROS]] or the Lego Mindstorms EV3 via [[http://www.phratch.com/jetstorm|JetStorm]], download the respective bridge packages, as detailed below. ** Installing LRP From Source ** Alternatively, you can install LRP from source, do-it of the following in a playground: Gofer it smalltalkhubUser: 'jfabry' project: 'LiveRobotProgramming'; configuration; loadDevelopment === ROS Support === ROS integration is provided thanks to [[http://car.mines-douai.fr/category/software/pharos/|PhaROS]]. You can download [[https://dl.dropboxusercontent.com/u/31426460/pharo-LRP.tar.gz|a working image with LRP and PhaROS]] that is relatively up-to-date. This image is ready to work, you will have a new menu entry in the World menu that opens the LRP UI. (Click on the background to get the World menu.) When the interpreter is opened, a separate UI allows for making subscriptions to topics and declaring topics on which to publish. For example code that uses ROS we refer to [[https://github.com/jfabry/LiveRobotProgramming/tree/master/ROS%20Examples|our GitHub repository.]] ** From Source ** We will assume you are using a Ubuntu linux, as this is the standard OS for ROS. To install from source, first [[http://car.mines-douai.fr/2014/06/how-to-install-pharos/|PhaROS should be installed]] instead of Pharo. Note that there may be PhaROS installation issues((Some installation issues on Indigo may be fixed using the following magic. First ''sudo apt-get install libX11-6:i386 libxext6:i386 lib32z1 libasound2:i386 libasound2-plugins:i386 libssl0.9.8:i386 libfreetype6:i386'' then (for Indigo) ''source /opt/ros/indigo/setup.bash'' then an alternate installation ''curl http://car.mines-douai.fr/scripts/PhaROS | sed 's/libgl1-mesa-glx:i386/ /' | bash'')). LRP should then be installed from source in the package of choice (see above). LRP requires the Cairo graphics library, which is may not be installed. To install, do a ''sudo apt-get install libcairo2:i386'' in a terminal. The ROS bridge is installed as follows. Note: **both** Gofer load directives are required. Gofer it smalltalkhubUser: 'jfabry' project: 'LiveRobotProgramming'; package: 'LiveRobotics-Bridge-PhaROS'; load. Gofer it smalltalkhubUser: 'jfabry' project: 'LiveRobotProgramming'; package: 'LiveRobotics-UI-PhaROS'; load Next time the LRP interpreter is opened the ROS bridge UI will open, asking for the name of the class that represents the current package. If you have problems installing PhaROS, you can bypass the main installation by downloading only the PhaROS API for Pharo. Gofer it smalltalkhubUser: 'CAR' project: 'PhaROS'; configuration; load After installing PhaROS, you can install the LRP ROS bridge (see above). === Parrot AR.Drone Support === The Parrot AR.Drone 2 is also supported in LRP. To use it, get the Pharo API from [[http://smalltalkhub.com/#!/~CaroHernandez/ArDronePharo]], the documentation of the API is also at this repository and the LRP bridge can also be found there. Example LRP code for the drone [[https://github.com/carolahp/ARDroneLRP-Tests|is available on GitHub]]. === Lego Mindstorms EV3 Support === The Mindstorms EV3 can be remote-controlled over WiFi if the Lego-supported USB WiFi adapter is fitted (a [[http://education.lego.com/en-us/lego-education-product-database/mindstorms-ev3/45544-lego-mindstorms-education-ev3-core-set/|"NETGEAR WiFi dongle WNA1100 Wireless-N 150"]]). Note: only **this exact** adapter is supported by default. Connecting to a WiFi network is relatively straightforward, and there are also tutorials on-line [[http://www.monobrick.dk/guides/how-to-establish-a-wifi-connection-with-the-ev3-brick/|e.g. the "Connecting the EV3 to a network" section of this page]]. First install JetStorm (instructions adapted from [[http://www.phratch.com/jetstorm/jetstorminstall|the JetStorm install page]]). Gofer it smalltalkhubUser: 'JLaval' project: 'JetStorm'; configuration; loadBleedingEdge. Second install the JetStorm bridge as follows: Gofer it smalltalkhubUser: 'jfabry' project: 'LiveRobotProgramming'; package: 'LiveRobotics-Bridge-JetStorm'; load Next time the LRP interpreter is opened the Jetstorm bridge UI will open, asking for the IP address of the EV3 and then setting up the connection. **Note that** the proprietary Lego protocol sadly does not fulfill usual Lego robustness standards and connection setup easily fails on a busy network. One way to see whether the EV3 is broadcasting its availability is by using the tcpdump command, as shown in the example below, for a brick that has an IP address of 192.168.0.87 : Makivi:~ jfabry$ sudo tcpdump src 192.168.0.87 and udp Password: tcpdump: data link type PKTAP tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on pktap, link-type PKTAP (Packet Tap), capture size 65535 bytes 15:51:35.666729 IP 192.168.0.87.54772 > 192.168.0.255.nati-dstp: UDP, length 67 15:51:40.582201 IP 192.168.0.87.54772 > 192.168.0.255.nati-dstp: UDP, length 67 15:51:45.600016 IP 192.168.0.87.54772 > 192.168.0.255.nati-dstp: UDP, length 67 15:51:50.617837 IP 192.168.0.87.54772 > 192.168.0.255.nati-dstp: UDP, length 67 15:51:55.637489 IP 192.168.0.87.54772 > 192.168.0.255.nati-dstp: UDP, length 67 A message should be received at least every 5 seconds (as above) for the connection to be set up successfully. If you receive messages at a lower frequency, the network is probably too overloaded or the WiFi radio spectrum too noisy for the connection to work. Note that if you receive no packets at all, this may be because your base station filters out these packets. For example code that uses the EV3 we refer to [[https://github.com/jfabry/LiveRobotProgramming/tree/master/Mindstorms%20Examples|our GitHub repository.]] ==== Bug Tracker ==== The bug tracker for LRP is [[https://github.com/jfabry/LiveRobotProgramming/issues|hosted by GitHub]].