понедельник, 25 сентября 2017 г.

Class meta annotations

In Pharo language we can annotate methods with meta information using special syntax:
MyClass>>someMethod
  <mySpecialPragmaWith: firstArg and: secondArg>
  "method logic"
The meta information is represented by pragma instances. It can be retrieved from methods:
(MyClass>>#someMethod) pragmas
And it can be queried globally from the system:
Pragma allNamed: #mySpecialPragmaWith:and: in: MyClass
You can read details about pragmas here: Pragmas: Literal Messages as Powerful Method Annotations.

Interesting that method pragmas can be used to implement class annotations without introducing special language constructs for them. Many libraries do it without explicit mention of annotations.

Usually the trick is class side method which is annotated with special conventional pragma which arguments are used as class meta information. So you query the system for this pragma and from methods you get classes and from pragma you get metadata.

The problem here is that pragma is quite simple and therefore it is quite restrictive:
  • It is always an instance of class Pragma. So you can not add specific behaviour to it.
  • Pragma arguments are literal objects. So you can not use anything complex in pragmas. And again you can not get metadata with specific behaviour
To address this problems you can return special object from "class annotating" methods. It will allow you to add any behaviour to it and instantiate it together with any other object without restrictions. In that case to query annotations you will evaluate found pragma methods and collect result. In addition you will probably need cache results if annotations are part of some frequent processing logic. And for cache you will need some invalidation logic because "class annotating" methods can be removed or added.

So many libraries implement described mechanizms in one or another way. And they all duplicate implementation of class annotations in one or another way.

For example in Commander users annotate command classes with activators which represent the way how access and execute commands (shortcut, context menu, etc..). Activators are attached as class side methods with pragma #commandActivator. Each method returns activator instance.

Calypso browser has similar mechanism to declare what tools should be in the browser as tabs. Each tool defines class side methods which return the context of browser where it should be shown.

Generally class annotations are very known and useful concept. In Java and C# they are used everywhere: serialization, ORM, web services, whatever.

So I decided introduce class annotations as reusable mechanizm in Pharo. You can look at first version on github https://github.com/dionisiydk/ClassMetaAnnotation. Interesting that it is just one class with two method extensions. Main logic is based on existing PragmaCollector. That's why it is so small library.

So in my implementation every annotation is subclass of ClassMetaAnnotation. To attach it to classes you should create new class side method which will return the instance of annotation. This method should be marked with pragma #classMetaAnnotation:
MyClass class>>specialAnnotation
  <classMetaAnnotation>
  ^MySpecialAnnotation new
Then you can query annotations using one of the following methods:
You can ask concrete annotation class for all declared instances:
MySpecialAnnotation declaredInstances
And you can ask given class for all attached annotations:
MyClass metaAnnotations
All annotations are cached. So it is cheap to query them.

Now I start adopt Calypso and Commander to use this new mechanizm. And I think there are many places where class annotations will simplify the system.

Interesting that it really took time to realize that the things which Commander and Calypso implement are actually annotation logic. Thank's Marcus Denker who helps me with this.

пятница, 25 августа 2017 г.

PharoThings. A live programming IoT platform based on Pharo

I am glad to announce the project PharoThings which brings the live programming environment into IoT domain. 

It includes:
  • development tools to lively program, explore and debug remote boards (based on TelePharo)
  • board modeling library which simplifies board configuration
    • Raspberry driven by WiringPi library
    • Arduino driven by Firmata, soon
    • Beaglebone, soon
Follow github page and videos to get a feeling of this project:
Now PharoThings is in beta stage together with documentation and videos. I would like any feedback on how to improve them

среда, 23 августа 2017 г.

PharmIDE is renamed to TelePharo and moved to github

This time to talk about PharmIDE migration.

In the team we finally agreed on the end name for project. It is now TelePharo.
Prefix "tele" means some action on or by distance. And this is what the project is about: to work remotely with Pharo images.

With new name project was moved to github repository https://github.com/dionisiydk/TelePharo with all dependencies:


According to the new name classes were renamed with the new prefix "Tlp".

TelePharo brings tiny bug fixes and new tools comparing to previous project.

Import remote code changes


Now you are able to import remote code changes (thank's Norbert for the script):
remotePharo applyChangesToClient
Underhood it loads all Epicea events from remote image and applies them into the local image. Then you just need commit the code into source repository.

Remote process browser


The new remote tool is added. You can browse processes which are running on remote image:
remotePharo openProcessBrowser

It is very initial version of ProcessBrowser based on Calypso. It will be improved in future. Now it allows explore and terminate processes on remote Pharo image.

Other improvements


  • Playground/inspector scripts are safe for disconnection from remote side

On remote side script can use remote variables (from remote playground). And now if communication is broken (bad connection) it will use cached values and will not break. It means that you are able to run processes on remote side using playground variables. And these processes will continue work when you disconnect.

  • Command line option "startServerOnPort" is safe when image was saved with running server

In past this option broke startup of image which included running server.
Also at the end of startup TelePharo prints actual server port into console

  • Better remote image save

Now expression "remotePharo saveImage" waits until remote side will be saved and connection will be recovered.

  • Some optimisations on remote communication

That's all. Feel free to fork TelePharo and report bugs. 

P.S. For those who skipped previous project here is the link. Notice that valid instructions are now at TelePharo project page and classes are now renamed

пятница, 28 июля 2017 г.

Calypso update: many improvements and migration to GitHub

New version 0.7.9 is done. Calypso is now hosted on https://github.com/dionisiydk/Calypso.
To load it use following script:
Metacello new
  baseline: 'Calypso';
  repository: 'github://dionisiydk/Calypso';
  load
Feel free to fork it and pull requests to dev branch.

Main improvement is navigation history:


It remembers table selections, browser modes and active tabs. You can switch browser to class hierarchy and go back using shortcut or button.
Another cool feature is across window navigation. When you spawn new browser "go back" command will close this window and focus initial one. "Go forward" in that case will open closed window again. Video below shows it in live:


You can notice that now method editor allows edit protocol and package just in place:


You don't need anymore create protocol in advance and select it. Just type it directly in method editor or choose extending package if you want.
These editors are placed in status bar which can be filled by tools and browser plugins. For example critic plugin shows analysis indication in status bar (look at the bottom of first image).

SUnit plugin is also improved. It extends toolbar of method browser with two buttons. One button runs all tests from method list. Another button runs all failures:


Full browser adds special "failed tests" group which appears when failure exists:


With selected group you can open all tests in method browser by context menu command.

Extra tab with #setUp method is shown when test class is selected. It is suitable tool which provide fast access to setup logic of test case. And you can see test and setup at once:


Now other method groups in pictures:

  • Abstract methods
  • Overridden methods
  • Overrides methods

  • Methods which should be implemented. They are defined as abstract in superclasses


  • Breakpoints


  • Deprecated methods
  • Critiques. It is most interesting group which collects all problem methods by evaluating critic analysis in background 


The next main feature will be scoped refactoring. Idea is to extend current change browser


with scope selection tool in the way similar to method browser:


New change browser is also needed to remove dependency from Nautilus. 
Soon we will integrate Calypso in Pharo 7 and removing Nautilus code will be another important task.

понедельник, 22 мая 2017 г.

PharoDays2017. Calypso presentation

Another Pharo year is gone. I made presentation about Calypso browser at PharoDays2017.
You can look at slides here:


And try it lively by:
Metacello new
  baseline: 'Calypso';
  repository: 'github://dionisiydk/Calypso';
  load.

#ClySystemBrowser asClass open

вторник, 11 апреля 2017 г.

Commander: command pattern library

Commander is a new Pharo library which models application actions as first class objects.
Every action is implemented as separate command class (subclass of CmdCommand) with #execute method and all state required for execution.
Commands are reusable objects and applications provide various ways to access them: shortcuts, context menu, buttons, etc.. This information is attached to command classes as activator objects. Currently there are three types of activators:
  • CmdShortcutCommandActivator
  • CmdContextMenuCommandActivator
  • CmdDragAndDropCommandActivator
Activators are declared in command class side methods marked with pragma #commandActivator. For example following method will allow RenamePackageCommand to be executed by shortcut in possible system browser:  
RenamePackageCommand class>>packageBrowserShortcutActivator
  <commandActivator>
  ^CmdShortcutCommandActivator by: $r meta for: PackageBrowserContext
And for context menu:  
RenamePackageCommand class>>packageBrowserMenuActivator
  <commandActivator>
  ^CmdContextMenuCommandActivator byRootGroupItemFor: PackageBrowserContext
Activators are always declared with application context where they can be applied (PackageBrowserContext in example). Application should provide such contexts as subclasses of CmdToolContext with information about application state. Every widget can bring own context to interact with application as separate tool. For example system browser shows multiple panes which provide package context, class context and method context. And depending on context browser shows different menu and provides different shortcuts.
To support activators command should implement several methods:  
  • canBeExecutedInContext: aToolContext
By default it returns true. But usually commands query context for specific information. For example RenamePackageCommand requires package and it defines this method as:  
RenamePackageCommand>>canBeExecutedInContext: aToolContext
   ^aToolContext isPackageSelected
  • prepareFullExecutionInContext: aToolContext
In this method command should retrieve all state required for execution. It can also ask user for extra data. For example RenamePackageCommand retrieves package from context and asks user for new name:  
RenamePackageCommand>>prepareFullExecutionInContext: aToolContext
    package := aToolContext selectedPackage.
    newName := UIManager default
        request: 'New name of the package'
        initialAnswer: package name
        title: 'Rename a package'.
    newName isEmptyOrNil | (newName = package name) ifTrue: [ ^ CmdCommandAborted signal ]
To break execution command can raise CmdCommandAborted signal.  
  • applyResultInContext: aToolContext
Purpose of this method is to be able interact with application when command completes. For example if user creates new package from browser then at the end of command browser should open created package:  
CreatePackageCommand>>applyResultInContext: aToolContext
    aToolContext showPackage: resultPackage
Commands are supposed to be reusable for different contexts and these methods should be implemented with that in mind. They should not discover internal structure of contexts.
Specific context can override activation methods and send own set of messages to command. For example:  
SpecialContextA>>allowsExecutionOf: aCommand
     ^aCommand canBeExecutedInSpecialContextA: self

SpecialContextA>>prepareFullExecutionOf: aCommand
  aCommand prepareFullExecutionInSpecialContextA: self

SpecialContextA>>applyResultOf: aCommand
  aCommand applyResultInSpecialContextA: self
By default CmdCommand can implement them with standard context methods. And only particular commands will override them specifically:  
CmdCommand>>prepareFullExecutionInSpecialContextA: aSpecialContextA
  self prepareFullExecutionInContext: aSpecialContextA

SomeCommand>>prepareFullExecutionInSpecialContextA: aSpecialContextA
  "special logic to prepare command for execution"
Different kind of activators extend commands with new protocol to support them. For example context menu activator add building method to command:  
command fillContextMenu: aMenu using: anActivator
By default it just creates item morph and allow subclasses to define default label and icon:
  • defaultMenuItemName
  • setUpIconForMenuItem: aMenuItemMorph
But subclasses can override build method to represent themselves differently. For example they can create item morph with check box.
The way how concrete type of activator hooks into application is responsibility of application. For example to support shortcuts based on commands application should define specific kmDispatcher for target morphs:  
YourAppMorph>>kmDispatcher
  ^ CmdKMDispatcher attachedTo: self
with supporting method:  
YourAppMorph>>createCommandContext
  ^YourAppContext for: self
If application wants context menu based on commands then it needs to hook into context menu part of application and ask activator to build menu:  
menu := CmdContextMenuCommandActivator buildMenuFor: anAppMorph inContext: aToolContext
In future Commander will provide deep integration with UI. And many things will work automatically.
To load code use following script:  
Metacello new
  baseline: 'Commander';
  repository: 'github://dionisiydk/Commander';
  load.
Detailed documentation can be found here

четверг, 16 февраля 2017 г.

Calypso navigation model

This time I am going describe Calypso navigation model.

In Calypso users query environment for specific set of objects.
First you need environment instance. There is global one for current image:
env := ClyNavigationEnvironment currentImage
It is navigation environment which is created over some system environment. In this case it is current image:
ClySystemEnvironment currentImage
System environment models the image. It includes package manager, globals and system announcer. And navigation environment provides interface to query information from it. It organizes cache of all queries.  If you will browse senders of message #do: same result will be returned for second call. Cache optimizes performance and memory usage during navigation.
You can use your own navigation instance instead of global one:
env := ClyNavigationEnvironment over: ClySystemEnvironment currentImage
External libraries can provide own system environment and Calypso will be able browse it. For example new version of Ring allows browse code repository with all Calypso features.

For next steps you need a scope where you will look at environment objects.
It can be scope of full system:
systemScope := env systemScope
Or it can be scope of concrete objects:
packageScope := env selectScope: ClyPackageScope of: {Point package}. 
classScope := env selectScope: ClyClassScope of: {Point. Collection}
With scope you can evaluate queries:
packageScope query: ClySortedClasses 
classScope query: ClySortedMethods
packageScope query: (ClyMessageSenders of: #(do: x))
classScope query: (ClyMessageImplementors of: #(select: y:)) 
Any query returns instance of cursor which provides stream access to result (details below).
Result is represented by requested ClyEnvironmentContent subclass. In first example it is instance of ClySortedClasses which is sent as argument. Query method accepts query instance (subclasses of ClyEnvironmentQuery) or compatible object which implements #asEnvironmentQuery message. And class of result itself plays role of most trivial query ClyAllItemsQuery. This query is responsible to return all objects accessible from given scope in requested form. For example you can query all classes in hierarchical form:
packageScope query: ClyHierarchicallySortedClasses
Any query is created with requested content or defines default one.  In example senders and implementors use sorted methods by default. But they can use different:
classScope query: (ClyMessageSenders of: #(do:) as: ClyHierarchicallySortedMethods)
As was mentioned above actual result of #query: is cursor object, the instance of ClyEnvironmentCursor:
cursor := classScope query: ClySortedMethods.
Cursor provides stream access to requested content items:
cursor currentItem.
cursor nextItem.
cursor moveTo: itemPosition. 
cursor retrieveAll.
Returned items are not raw objects like methods or classes. Instead they are instances of ClyEnvironmentItem which wraps actual object. Items extend objects with arbitrary set of properties. For example if class has abstract method it can be marked with "abstract property". If method is overridden in subclasses it can be marked by "overridden property". Then tools use these properties to provide specific representation for items. For example browser can show abstract classes with italic font and it can show special icon nearly overridden method.

Computation of properties is slow. Imaging that you look at all classes in system and for each class you want abstract property. It will require scanning almost all methods in system.
Calypso solves this problem in two ways:
  • All queries are cached. Computation will be performed only once
  • Properties are computed lazily when objects are really used.  For example they computed for objects which are shown to user, for only visible part of them.
Lazy computation is hidden from users by cursor instance. Cursor asks observed environment content to resolve small part of items according to current position. Then cursor uses resolved part as items cache. And new portion of items are resolved by demand.

This logic provides important optimization for remote scenario where observed content is remote object. In this case only used part of items is transferred over network. And when next part is needed it is loaded to client. It makes remote browser very fast because only visible part of packages, classes and methods are sent over network and all required information is available in few requests.

Now let's play with example cursor. It can be opened in table morph:
cursor := packageScope query: ClySortedClasses.
dataSource := ClyCollapsedDataSource on: cursor.
table := FTTableMorph new.
table
extent: 200 @ 400;
dataSource: dataSource;
openInWindow. 

Cursor provides query API to evaluate new queries with modified parameters.
For example you can ask different kind of result for original query:
cursor := cursor query: ClyHierarchicallySortedClasses


Or you can evaluate new query with extra package:
cursor := cursor queryContentInScopeWith: #'AST-Core' asPackage.

Or you can evaluate new query without existing package:
cursor := cursor queryContentInScopeWithout: #Kernel asPackage.

With cursor you can navigate to the children scope of particular items. For example you can query methods of selected classes:
cursor := cursor query: ClySortedMethods from: {Point. Array}.

You can also evaluate original query in different scope. It allows fetch class side methods of selected classes:
cursor := cursor queryContentInNewScope: ClyClassSideScope. 

All these queries never modify cursor state. They always return new cursor instance which points to new result.

On top of this navigation model Calypso implements tree structure. For example you can look at classes and methods in same table morph:
cursor := packageScope query: ClySortedClasses.
dataSource := ClyCollapsedDataSource on: cursor.
dataSource childrenStructure: { ClySortedMethods }.
table := FTTableMorph new.
table
extent: 200 @ 400;
dataSource: dataSource;
openInWindow

Tree structure can include many levels. For example packages -> class groups -> classes:
cursor := env systemScope query: ClySortedPackages.
dataSource := ClyCollapsedDataSource on: cursor.
dataSource childrenStructure: { ClySortedClassGroups. ClyHierarchicallySortedClasses}.
table := FTTableMorph new.
table
extent: 200 @ 400;
dataSource: dataSource;
openInWindow

Also it is possible to show hierarchical items with collapsing buttons:
cursor := packageScope query: ClyHierarchicallySortedClasses.
dataSource := ClyExpandedDataSource on: cursor.
table := FTTableMorph new.
table
extent: 200 @ 400;
dataSource: dataSource;
openInWindow

At the end let's implement new environment content RandomOrderedMethods:
ClyMethodsContent subclass: #ClyRandomOrderedMethods
instanceVariableNames: ''
classVariableNames: ''
package: 'Calypso-Example'
Each kind of environment content implements set of the building methods depending on what scope it supports. For example ClySortedMethods supports extracting methods from classes and method groups. ClySortedClasses supports extracting classes from packages and class groups.
When ClyAllItemsQuery is evaluated by given scope it asks requested content to be built from concrete objects. For package scope it will ask content for #buildFromPackages:. For class scope it will ask content for #buildFromClasses:.
For simplicity ClyRandomOrderedMethods will support only class scope:
buildFromClasses: classes
items := OrderedCollection new.
classes do: [ :eachClass |
eachClass localMethods do: [ :each |
items add: (ClyEnvironmentItem named: each selector with: each)] ].
items shuffle
Now look at methods in new form:
cursor := env systemScope query: ClySortedPackages.
dataSource := ClyCollapsedDataSource on: cursor.
dataSource childrenStructure: { ClySortedClasses. RandomOrderedMethods}.
table := FTTableMorph new.
table
extent: 200 @ 400;
dataSource: dataSource;
openInWindow

"Building" methods are required to be able evaluate "all items queries" with new implemented form. More advanced queries could require another code. Generally queries implement one method:
  • fetchContent: anEnvironmentContent from: anEnvironmentScope
where they dispatch processing to scope and environment content depending on their logic. For example to be able to use new random order for senders or implementors the method #buildFromMethods: should be implemented by content:
buildFromMethods: methods
items := methods shuffled collect: [ :each |
ClyEnvironmentItem named: each selector with: each]
If you open senders browser:
browser := ClyMethodBrowser browseSendersOf: #do:.

you can switch it to new order:
browser switchToResultContent: ClyRandomOrderedMethods

For more details look at code and read class comments.
Next time I will show how extend browser with new functions.

вторник, 24 января 2017 г.

PharmIDE: Pharo remote IDE to develop farm of Pharo images remotely

I am glad announce first version of PharmIDE project which is complete toolset for remote development of Pharo images. It includes:
  • remote debugger
  • remote inspector
  • remote playground
  • remote browser
Old project RemoteDebuggingTools is deprecated. But it could be used for Pharo 5 images. PharmIDE targets Pharo 6 or later and provides everything which was done in original project.

Server part of project should be installed on target image:
Metacello new
        smalltalkhubUser: 'Pharo' project: 'PharmIDE';
        configuration: 'PharmIDE';
        version: #stable;
        load: 'Server'
Then server should be started on port where client image can connect:
PrmRemoteUIManager registerOnPort: 40423
Image can be saved with running server. It will be automatically started when image restarts. Or you can use command line option for this:
./pharo PharoServer.image remotePharo --startServerOnPort=40423
On IDE image client side of project should be installed:
Metacello new
        smalltalkhubUser: 'Pharo' project: 'PharmIDE';
        configuration: 'PharmIDE';
        version: #stable;
        load: 'Client'
And then you can connect Pharo to remote image:
remotePharo := PrmRemoteIDE connectTo: (TCPAddress ip: #[127 0 0 1] port: 40423)
It registers local debugger and browser on remote image:

  • Any error on remote image will open debugger on client image with remote process stack
  • Any browser request on remote image will open browser on client image with remote packages and classes
  • User requests from server are redirected to client. Any confirm or inform requests from remote image will be shown on client. For example author name request will be shown on client image where user can type own name remotely.

With remotePharo instance you can evaluate scripts:
remotePharo evaluateAsync: [ [1/0] fork ].
remotePharo evaluate: [ 1 + 2 ] "==> 3".
remotePharo evaluate: [ 0@0 corner: 2@3 ] "==> aSeamlessProxy on remote rectangle".
Look at original project post for more details about scripting and inspecting. In this version inspector is improved:
  • PrintIt command shows correct #printString of remote object instead of 'aSeamlessProxy~'. 
  • "Workspace" variables are supported in inspector. You can save temp results in it.
  • Implementors/senders shortcuts will open local browser on remote environment. Any subsequent navigation will be bound to remote image.
Remote playground and remote browser support same set of features. All text editors support same commands like in local environment.

To open browser or playground use:
remotePharo openPlayground.
remotePharo openBrowser
Now #debugIt command and refactorings are not working remotely but they will be supported in future.

Now look at PharmIDE in action:



Update. Project was renamed to TelePharo and moved to github. Details are here http://dionisiydk.blogspot.fr/2017/08/pharmide-is-renamed-to-telepharo-and.html



четверг, 12 января 2017 г.

Calypso update: MethodBrowser and better UI

I glad release new version of Calypso. It includes method browser which makes Calypso complete toolset for system navigation.

Also new version contains many UI changes:
  • white toolbar (according to theme background color)
  • separators between radio button groups
  • clickable labels instead of buttons


Method browser has many new cool features:
  • Flat and hierarchy view modes. First shows methods as flat list sorted by selectors. And last sorts methods according to class hierarchy 
  • Scopes. You can switch between package, class or system scopes to filter methods. Scopes are inherited from initial tool which opens browser
  • "Child" method browsers are opened in tabs. Click on senders/implementors button do not open new window but instead it shows result in new window tab

To make Calypso default tool evaluate:
ClyBrowser beAllDefault
And little live video: