вторник, 12 декабря 2017 г.

New Calypso version 0.8 is out

Finally I finish major Calypso changes. It is a big refactoring on navigation model and browser pluggability.

For the model part the biggest change is composition of queries and scopes. 
For example string lookup in class comments and method sources looks like:

(ClyClassComments withSubstring: 'test') , (ClyMethodSources withSubstring: 'test')

Query of all variables accessible from the class can be done with:

ClyAllVariables fromAll: {ClyClassScope. ClySuperclassScope} of: Morph in: environment

In the browser these compositions are created from some concrete scope using conversion methods. For example inherited scope can be constructed with:

classScope , (classScope asScope: ClySuperclassScope)

And full hierarchy scope looks like:

classScope
     , (classScope asScope: ClySuperclassScope)
     , (classScope asScope: ClySubclassScope)

The next important feature is async queries. Now it is possible to evaluate any query in background. Just convert given query using #async message:

asyncQuery := aQuery async

Browser provides nice visualisation for execution process. It is very simple rotating icon. But real progress indication can be implemented later. Try following script to see it in action:

ClyQueryBrowser openOn: (
    (ClyClassComments withString: 'test'), (ClyMethodSources withString: 'example')) async.

Composition and async queries were required to support advanced search functions like "method source with it". Also critic plugin is now based on them instead of complicated internal implementation for background analysis. 
Nice detail about async queries: they are not bound to global progress bar. The animation is shown in the browser and the query not blocks UI.

With help of new composed design I was able to redo all method groups using queries. Normally if you want new method group you do not need new subclass of group. You only implement new query and group provider instantiates a group on it. For example breakpoints provider is implemented with following methods:

ClyBreakMethodGroupProvider>>createMethodQueryFrom: aClassScope
^ClyActiveBreakpoints from: aClassScope

ClyBreakMethodGroupProvider>>createMethodGroupFor: aMethodQuery from: aClassScope
^ClyMethodGroup named: 'breakpoints' priority: 8.4 on: aMethodQuery 

Interesting that in case when provider returns async query it will be automatically represented by group item with progress animation. It is how critic method group is now implemented. 

Another big change is the result of query. If you will execute query it will return the instance of ClyQueryResult subclass. Previously it was always cursor object which is now opened by demand from given result. 
Purpose of different kinds of result is to format items in different forms. There is ClySortedQueryResult with sort function as parameter (that's why I worked on SortFunction). And there are many kinds of result which return hierarchical browser items.
You can get any query with required result:

newQuery := classQuery withResult: (ClySortedQueryResult using: #name ascending)
newQuery := classQuery withResult: ClySubclassHierarchy new asQueryResult 

By default any query is created with ClyRawQueryResult which represents retrieved items without any formatting. So by default any query returns raw methods, classes and packages. It allows in future to replace current SystemNavigation with Calypso query model.

On the browser part there are many renames. As you noticed ClyQueryBrowser replaced method browser with ability to show mixed query result. In example it shows class comments and methods together in single list. It also able to show classes. Interesting that hierarchical view and sorted view modes are working in all these cases.
The ClySystemBrowser is renamed to ClyFullBrowser.

There are a lot of other internal changes and bug fixes. Here is short list of them:
  • better class hierarchies implementation
    • merged hierarchies
      • mixed traits and subclasses/superclasses relation
    • inverse relation hierarchy
      • pluggable order for variables view
      • pluggable order for class visibility view
  • class annotations for browser pluggability
    • table decorators are annotated by browser context where they are applied
    • browser tools are annotated by browser context where they are activated as tabs
  • new Commander based on class annotations
  • repackaged code
  • many classes are renamed
  • traits are in separate plugin
  • browser tabs subscribe on system changes by themselves. It fixes sync bug when tabs are not updated after system changes
  • search packages like in Nautilus with default persistent filter
  • a lot of new system queries
Now I start document new design and I will post more details soon 

понедельник, 25 сентября 2017 г.

Class annotations

In Pharo language we can annotate methods with meta information using special syntax:
MyClass>>someMethod
  <mySpecialPragmaWith: firstArg and: secondArg>
  "method logic"
The meta information is represented by pragma instances. It can be retrieved from methods:
(MyClass>>#someMethod) pragmas
And it can be queried globally from the system:
Pragma allNamed: #mySpecialPragmaWith:and: in: MyClass
You can read details about pragmas here: Pragmas: Literal Messages as Powerful Method Annotations.

Interesting that method pragmas can be used to implement class annotations without introducing special language constructs for them. Many libraries do it without explicit mention of annotations.

Usually the trick is class side method which is annotated with special conventional pragma which arguments are used as class meta information. So you query the system for this pragma and from methods you get classes and from pragma you get metadata.

The problem here is that pragma is quite simple and therefore it is quite restrictive:
  • It is always an instance of class Pragma. So you can not add specific behaviour to it.
  • Pragma arguments are literal objects. So you can not use anything complex in pragmas. And again you can not get metadata with specific behaviour
To address this problems you can return special object from "class annotating" methods. It will allow you to add any behaviour to it and instantiate it together with any other object without restrictions. In that case to query annotations you will evaluate found pragma methods and collect result. In addition you will probably need cache results if annotations are part of some frequent processing logic. And for cache you will need some invalidation logic because "class annotating" methods can be removed or added.

So many libraries implement described mechanizms in one or another way. And they all duplicate implementation of class annotations in one or another way.

For example in Commander users annotate command classes with activators which represent the way how access and execute commands (shortcut, context menu, etc..). Activators are attached as class side methods with pragma #commandActivator. Each method returns activator instance.

Calypso browser has similar mechanism to declare what tools should be in the browser as tabs. Each tool defines class side methods which return the context of browser where it should be shown.

Generally class annotations are very known and useful concept. In Java and C# they are used everywhere: serialization, ORM, web services, whatever.

So I decided introduce class annotations as reusable mechanizm in Pharo. You can look at first version on github https://github.com/dionisiydk/ClassAnnotation. Interesting that it is just one class with two method extensions. Main logic is based on existing PragmaCollector. That's why it is so small library.

So in my implementation every annotation is subclass of ClassAnnotation. To attach it to classes you should create new class side method which will return an instance of the annotation. This method should be marked with pragma #classAnnotation:
MyClass class>>specialAnnotation
  <classAnnotation>
  ^MySpecialAnnotation new
Then you can query annotations using one of the following methods:
You can ask concrete annotation class for all registered instances:
MySpecialAnnotation registeredInstances
And you can ask given class for all attached annotations:
MyClass classAnnotations
All annotations are cached. So it is cheap to query them.

Now I start adopt Calypso and Commander to use this new mechanizm. And I think there are many places where class annotations will simplify the system.

Interesting that it really took time to realize that the things which Commander and Calypso implement are actually annotation logic. Thank's Marcus Denker who helps me with this.

пятница, 25 августа 2017 г.

PharoThings. A live programming IoT platform based on Pharo

I am glad to announce the project PharoThings which brings the live programming environment into IoT domain. 

It includes:
  • development tools to lively program, explore and debug remote boards (based on TelePharo)
  • board modeling library which simplifies board configuration
    • Raspberry driven by WiringPi library
    • Arduino driven by Firmata, soon
    • Beaglebone, soon
Follow github page and videos to get a feeling of this project:
Now PharoThings is in beta stage together with documentation and videos. I would like any feedback on how to improve them

среда, 23 августа 2017 г.

PharmIDE is renamed to TelePharo and moved to github

This time to talk about PharmIDE migration.

In the team we finally agreed on the end name for project. It is now TelePharo.
Prefix "tele" means some action on or by distance. And this is what the project is about: to work remotely with Pharo images.

With new name project was moved to github repository https://github.com/dionisiydk/TelePharo with all dependencies:


According to the new name classes were renamed with the new prefix "Tlp".

TelePharo brings tiny bug fixes and new tools comparing to previous project.

Import remote code changes


Now you are able to import remote code changes (thank's Norbert for the script):
remotePharo applyChangesToClient
Underhood it loads all Epicea events from remote image and applies them into the local image. Then you just need commit the code into source repository.

Remote process browser


The new remote tool is added. You can browse processes which are running on remote image:
remotePharo openProcessBrowser

It is very initial version of ProcessBrowser based on Calypso. It will be improved in future. Now it allows explore and terminate processes on remote Pharo image.

Other improvements


  • Playground/inspector scripts are safe for disconnection from remote side

On remote side script can use remote variables (from remote playground). And now if communication is broken (bad connection) it will use cached values and will not break. It means that you are able to run processes on remote side using playground variables. And these processes will continue work when you disconnect.

  • Command line option "startServerOnPort" is safe when image was saved with running server

In past this option broke startup of image which included running server.
Also at the end of startup TelePharo prints actual server port into console

  • Better remote image save

Now expression "remotePharo saveImage" waits until remote side will be saved and connection will be recovered.

  • Some optimisations on remote communication

That's all. Feel free to fork TelePharo and report bugs. 

P.S. For those who skipped previous project here is the link. Notice that valid instructions are now at TelePharo project page and classes are now renamed

пятница, 28 июля 2017 г.

Calypso update: many improvements and migration to GitHub

New version 0.7.9 is done. Calypso is now hosted on https://github.com/dionisiydk/Calypso.
To load it use following script:
Metacello new
  baseline: 'Calypso';
  repository: 'github://dionisiydk/Calypso';
  load
Feel free to fork it and pull requests to dev branch.

Main improvement is navigation history:


It remembers table selections, browser modes and active tabs. You can switch browser to class hierarchy and go back using shortcut or button.
Another cool feature is across window navigation. When you spawn new browser "go back" command will close this window and focus initial one. "Go forward" in that case will open closed window again. Video below shows it in live:


You can notice that now method editor allows edit protocol and package just in place:


You don't need anymore create protocol in advance and select it. Just type it directly in method editor or choose extending package if you want.
These editors are placed in status bar which can be filled by tools and browser plugins. For example critic plugin shows analysis indication in status bar (look at the bottom of first image).

SUnit plugin is also improved. It extends toolbar of method browser with two buttons. One button runs all tests from method list. Another button runs all failures:


Full browser adds special "failed tests" group which appears when failure exists:


With selected group you can open all tests in method browser by context menu command.

Extra tab with #setUp method is shown when test class is selected. It is suitable tool which provide fast access to setup logic of test case. And you can see test and setup at once:


Now other method groups in pictures:

  • Abstract methods
  • Overridden methods
  • Overrides methods

  • Methods which should be implemented. They are defined as abstract in superclasses


  • Breakpoints


  • Deprecated methods
  • Critiques. It is most interesting group which collects all problem methods by evaluating critic analysis in background 


The next main feature will be scoped refactoring. Idea is to extend current change browser


with scope selection tool in the way similar to method browser:


New change browser is also needed to remove dependency from Nautilus. 
Soon we will integrate Calypso in Pharo 7 and removing Nautilus code will be another important task.