Architecture Hexagonale : ne vous perdez pas à droite

Fan de l’architecture hexagonale et de ses promesses depuis très longtemps, j’ai souvent passé du temps à la comprendre, l’expliquer mais aussi à démystifier son implémentation. Ce pattern a d’ailleurs accompagné une grosse partie de mes premières mises en œuvre du DDD au travail. Mais comme beaucoup trop de développeuses et de développeurs, je me suis souvent arrêté aux cas simples dans mes explications, aux “happy path” où tout se passe bien. Dans ce premier article d’une mini-série à venir, je souhaite évoquer cette fois quelques détails d’implémentation ainsi qu’un piège dans lequel il est très facile de tomber. L’idée étant de vous faire gagner un peu de temps lors de vos mises en œuvre de ce pattern d’architecture toujours aussi incroyablement utile.

Un Design-smell pour votre hexagone

Une fois n’est pas coutume, on va parler aujourd’hui d’un design-smell ou d’un anti-pattern : l’adaptateur de droite non cohésif. Si vous ne connaissez pas encore l’architecture hexagonale, je vous suggère de commencer par la lecture de mon précédent post sur le sujet qui récapitule tout ça ( http://tpierrain.blogspot.com/2016/04/hexagonal-layers.html). Car pour expliquer l’anti-pattern adaptateur de droite non cohésif, on va juste rappeler ici quelques fondamentaux au sujet de l’architecture hexagonale :

• Pour interroger le métier (le code du domaine), on s’adresse à un port que l’on appelle « driver port » ou « left-side port »
• Pour que le code du domaine puisse faire son travail, celui-ci peut avoir besoin d’utiliser des systèmes tierces (data store, web API, etc.). Pour ce faire, le code métier reste au niveau d’expressivité métier en interrogeant un ou plusieurs « driven ports » ou « right-side ports ».
• Un port est donc une interface (qui fait partie du Domaine) qui utilise une sémantique métier pour exprimer des demandes que l’on adresse à notre système (left-side) ou des actions/demandes que celui-ci effectue en cours de route pour arriver à ses fins (right-side). J’aime bien voir les ports comme des pont-levis pour aller et venir entre l’infra et le domaine et vice-versa (analogie que m’avait soufflée Cyrille MARTRAIRE il y a très longtemps)
• En termes de dépendance, la technique (le code d’infrastructure) connait et référence le métier. Enfin, soyons précis (au passage, merci à Alistair COCKBURN pour cette remarque pertinente) : seul le code d’infrastructure qui se trouve à gauche de l’hexagone connait le métier (à travers les portes d’entrée vers celui-ci que constituent les ports de gauche), pour interagir avec celui-ci.
• A l’inverse, le code métier (domaine) ne référence pas et ne connait pas le niveau technique. L’ensemble arrive à fonctionner au runtime grâce à l’inversion de dépendance que Gerard MESZAROS appelle lui des « configurable dependency » (ce qui se prête très bien au pattern d’Alistair)
• Un adapteur est un bout de code qui permet de passer d’un monde à l’autre au runtime (infra => domain ou domain => infra) et qui s’occupe notamment de convertir les structures de données d’un monde (ex : JSON ou DTO) à l’autre (ex : POJO ou POCO du domaine)
• Il y a 2 types d’adaptateurs : 
• Les « left-side adapters », qui ont une relation de type agrégation avec une instance de left-side port qu’ils utilisent
• Les « right-side adapters », qui implémentent une instance de right-side port

J’ai connu un adaptateur de droite une fois, qui avait dix fois plus de classes...

Vous aurez peut-être remarqué ici que je n’ai pas dit que les right-side adapters implémentaient plusieurs interfaces de right-side ports. C’est intentionnel.

Car si vous le faisiez, il y a de fortes chances pour que vous arriviez à l’anti-pattern que je veux évoquer ici : l’adaptateur de droite non cohésif.

De quoi s’agit-il ? C’est ce qui se passe lorsque vous confiez trop de responsabilités à un adaptateur de droite : vous risquez alors de déporter une partie de la logique de l’hexagone dans un de ses éléments périphérique (l’adaptateur). C’est ce qui se passe notamment lorsqu’un adapteur référence et utilise en direct un autre adapteur. C’est bien entendu quelque chose que je déconseille vivement (pour conserver toute la logique d’orchestration métier au niveau du domaine et pas la repartir au niveau du code d’infrastructure), mais vu que tout le monde tombe dans ce piège au moins une fois, ça vaut le coup d’être mentionné. Fuyez cette situation comme la peste (ou le COVID19).

 

Pire. Pour celles et ceux qui ne testeraient que le centre de l’hexagone (le code du domaine) en bouchonnant vos adaptateurs, vous passeriez à côté de pleins de code et de bugs potentiels qui se trouveraient nichés dans vos adaptateurs non testés. 

Il est en effet extrêmement facile de se bercer d’illusion en mettant plein d’implicites dans le comportement de vos stubs qui vont faire office d’adaptateurs. Vous y mettrez très souvent ce que vous pensez que les adaptateurs feront ou doivent faire sans pour autant vous donner la garantie que cela sera le cas. Cela vous donnera pleins de tests verts, mais avec des comportements potentiellement buggés en prod (quand vous utiliserez les vrais adaptateurs concrets).

Quelle stratégie de test au fait ?

J’ai déjà écrit et parlé sur la question mais comme je n’ai pour l’instant pas traduit en anglais mon article le plus spécifique sur le sujet, je vais juste clarifier ici quelques termes indispensables pour comprendre la suite de mon article.

Après plus de 15 ans de pratique du TDD au boulot, je suis arrivé au fil des années à une forme économe d’outside-in TDD qui convient très bien à 90% de mes projets. Par Outside-in, je veux dire que je considère mon système (souvent une API web) dans son ensemble et depuis l’extérieur. Un peu comme une grosse boite noire que je vais faire grandir et améliorer progressivement en écrivant des tests d’acceptation qui la pilote. Ceux-ci ne vont pas s’intéresser aux détails d’implémentation (ce qui est fait et codé à l’intérieur de la boite) mais uniquement au comportement métier de celle-ci en interagissant avec elle depuis l’extérieur. Mes tests d’acceptation sont donc des tests gros grains, qui testent la boite noire (ici, l’ensemble de l’hexagone) mais qui ne font pas appel à des vrais protocoles techniques ni a de vrais data store pour être le plus rapide possible (je suis accro au feedback court et utilise pour ça un outil qui fait tourner mes tests à chaque fois que je change une ligne de code).

A côtés de ces tests d’acceptation qui constituent 80% de ma stratégie, j’écris en général quelques tests unitaires grains-fins (les petits tests de la « double loop » quand j'en ai besoin), quelques tests d’intégrations, et des tests de stubs ou de contrats vis à vis de mes dépendances externes (pour détecter quand ils changent des choses). Dans la suite de cet article, je ne parlerai que de mes tests d’acceptation.

Le meilleur des compromis possibles

Cette parenthèse sur les types de tests étant fermée, revenons à notre problème qui est d’éviter d’écrire des tests incomplets de notre système, et qui vous donneraient l’impression que tout va bien dans votre hexagone alors qu’au runtime vous aurez des mauvaises surprises. 

Ma recommandation : écrivez des tests d’acceptation qui couvrent tout votre hexagone, à l’exception des I/Os en bout de chaine. Oui, vous m’avez bien entendu : je vous conseille d’écrire des tests d’acceptations qui utilisent vos adaptateurs concrets et pas des stubs. La seule chose qu’il vous faudra bouchonner (stubber) du coup, ce sont les appels réseaux, les accès disques ou base de données qui eux, sont faits par vos adaptateurs. Ce faisant, vous testez tout votre code, en mode assemblage complet (sans illusion ni mauvaise surprise en prod donc), mais blazing fast ;-) 

C’est toujours plus clair avec un Example

Oui, prenons un Example pour être encore plus clair (déformation professionnelle ;-)

Imaginez que vous ayez à coder un générateur de fichiers sitemap.xml pour le SEO d’une plate-forme web au contenu complètement dynamique et contrôlée par plusieurs ligne métiers (e-biz, marketing). Cet hexagone pourrait être une web API utilisée quotidiennement à côté du site web, pour régénérer tous les fichiers sitemaps de celui-ci suite à un appel de type POST (HTTP) par exemple. PS: Par contre, ne dites pas à ma mère que j’ai parlé de micro-service dans un de mes articles ;-)

Bon, et bien au lieu d’écrire des tests d’acceptation comme ça :

Je vous conseille plutôt d’écrire des tests d’acceptation dans lesquels vous allez inclure vos adaptateurs concrets et uniquement stubber leurs I/Os, comme ça : 

 

Peut mieux faire

Bien sûr, pour avoir des tests d’acceptations lisibles, concis (pas plus de 10/15 lignes) et expressifs (c.a.d. en voyant bien les valeurs transmises à vos bouchons), vous allez en général passer par des builders pour vos stubs, mocks, voire même pour l’assemblage de votre hexagone. Mais cela aurait été moins clair ici, pour mon exemple. Une version cible pourrait ressembler à ça :

Avertissements

On pourra m’objecter ici que mes tests d’acceptation sont un peu hybrides et qu’ils couvrent à la fois des considérations métiers mais aussi certaines plus techniques. Pas très orthodoxe tout ça ;-) 

Bon déjà moi l’orthodoxie ou le by-the-book, je m’en fous. Et puis c’est complètement transparent si vous faites en sorte que chacun de vos tests d’acceptation couvrent bien des comportements métiers de votre système (service/API/micro-service... choisissez ici le nom approprié). A vous de travailler l’expressivité de vos tests : le nom de ceux-ci, la simplicité et la brièveté de l’assemblage et du « arrange » en mode DSL par exemple.

Avant d’arriver par tâtonnement à cette nouvelle stratégie (cette année), je mettais uniquement en œuvre la stratégie de test mise en avant par nos copains Londoniens de Cucumber (Seb Rose, Steve Tooke et Aslak Hellesøy). Celle-ci vise à combiner :

-         Énormément de tests d’acceptation mais pour lesquels on stubbe l’intégralité des adaptateurs (des tests supers rapides comme les miens donc) 

-         Et puis quelques tests complémentaires de contrats, dans un autre projet. Ceux-ci vérifiant que nos stubs d’adaptateurs utilisés côté acceptation ont exactement le même comportement que les vrais adaptateurs concrets qui sont eux assemblés et livrés avec notre hexagone en prod. Ces derniers tests d’intégration sont donc beaucoup plus lents et exécutés plus rarement (essentiellement sur l’usine de dev, pas dans mon Runner automatique local et permanent ncrunch) mais c’est un compromis volontaire.

On avait donc une situation vraiment intéressante qui permettait d’avoir pleins de tests d’acceptation supers rapides qu’on faisait tourner en permanence, et d’être suffisamment en confiance sur la capacité de nos stubs d’adaptateurs de droite pour nous aider à atteindre cet objectif (car on testait aussi qu’ils sont fidèles à la réalité dans quelques scenarios de référence). 

J'en parle au passé ici, car je n’ai pas réellement réussi à avoir suffisamment confiance dans ce dispositif dans mes expériences avec différentes équipes. Ce qui nous posait régulièrement problème avec cette stratégie telle-quelle, c’était l’effort moindre mise par toute l’équipe dans ces derniers tests de contrats. On arrivait encore plus souvent que d’habitude à des situations avec beaucoup trop de happy path (des scénarios de base où tout se passe bien) dans ces tests de contrats. Ceux-ci ne couvraient pas assez de cas d’erreurs ou d’exceptions. Pour le dire autrement, nos tests d’acceptations sollicitaient nos stubs sur beaucoup plus de cas que ce qui était prévu pour eux dans nos tests de contrat les concernant.

Alors c’est vrai que les devs que nous sommes sont attirés par les happy path autant que les papillons de nuit le sont par une ampoule allumée. Ça semble être un de nos déterminisme ;-) Mais j’ai un peu observé ces situations pour essayer de comprendre ce qui n’allait pas et en suis arrivé à la conclusion que c’etait parce que ces tests d’intégrations -en plus d’être très « orientés plomberie »-sont beaucoup plus lents à faire tourner. C’est pour ça que les gens y apportaient moins d’attention. Un peu en mode : "de toute façon c’est un test d’intégration pour un stub... ça doit faire le job mais on ne va pas y passer trop de temps non plus". 

La conséquence de tout ça ? On testait en général beaucoup moins de combinaisons de cas dans ces tests de contrats de nos stubs beaucoup que ce dont on avait l’habitude de faire dans nos tests unitaires grains fin ou dans nos tests d’acceptation gros grains.

Ceci avait pour effet que cette association des deux types de tests n’était pas suffisante, pour arriver à attraper tous les bugs ou les problèmes de plomberie dans notre assemblage final. C’est donc pour ces raisons que j’en suis finalement arrivé à la stratégie de test que je vous ai présentée dans cet article, et qui inclus les adaptateurs concrets dans les tests d’acceptation. Par contre, j’utilise toujours cette stratégie de test de contrats pour les composants extérieurs ou des APIs tierce, c’est juste que maintenant je stubbe moins de choses, mes stubs ne couvrent plus qu’une partie très fine.  

Comme dernier avertissement, je dois aussi préciser que ma stratégie de test fonctionne incroyablement bien dans mon contexte. Je n’ai en effet jamais rencontré de cas comme celui d’Alistair, où j’avais besoin d’exposer la même API avec plusieurs technologie différentes à gauche  (HTTP, AMQP, MQTT...). J’ai en général une exposition unique de mon domaine : du HTTP dans une web API, ou du Aeron dans un service low latency, ou du RPC dans... (non je rigole, je déteste trop le modèle RPC ;-)

Je n’ai donc besoin que d’un seul adaptateur côté gauche. Ce qui m’évite d’être confronté au fait de devoir rejouer autant de fois mes tests d’acceptation que je j’aurai de technologies d’exposition différentes. Ca vaut le coup de le rappeler ici, j’utilise surtout l’architecture hexagonale parce qu’elle me permet de bien séparer mon code métier de mon code technique. Pas pour pouvoir interchanger facilement ma technologie d'exposition. 

Je sais que ce côté « plugins pour attaquer la même logique métier via des technos différentes » est une des forces de l’architecture hexagonale, mais c’est un cas que je n’ai pour l’instant rencontré que dans des katas. Je ne suis d’ailleurs pas le seul dans ce cas, car dans toutes les formations DDD que j’ai pu donner ou lors de discussions dans des conférences, j’ai eu de nombreux témoignages de gens qui venaient le voir en me disant la même chose : « j’utilise mes contrôleurs web (de web API) comme adaptateur de gauche, car je n’ai pas le cas d’une multi-exposition à gauche ».

Un dernier bénéfice pour la route

Ce qui est sûr en revanche, c’est que le fait de tout tester -sauf les I/Os- vous mettra dans tous les cas dans une situation confortable car :

• vos tests d’acceptation couvriront une code base très fidèle à la réalité de la prod
• votre harnais de tests d’acceptation vous permettra de refactorer sereinement votre base de code dans le cas où vous vous seriez trompés et auriez mis du comportement métier dans vos adaptateurs. Avec ces tests qui couvrent l’ensemble, la move pourra être fait sans risque.

Peut-être pourrions-nous trouver un moyen facile (autre que le pair/code review) pour éviter aux personnes les moins expérimentées de tomber dans ce piège. Pour l’instant cette capacité de refactorer facilement a posteriori si mes collègues ou moi-même nous sommes trompés (ou avons pris un peu de dette technique court-terme), était largement suffisante sur mes projets. 

En conclusion

1. Passez toujours par le centre de l’hexagone, ne connectez pas vos right-side adapters entre eux
2. Ne codez pas VOTRE logique métier ni VOTRE logique d’orchestration dans vos adaptateurs
3. Testez l’intégralité de votre hexagone (adaptateurs compris); ne bouchonnez que vos I/Os en bout de chaine utilisés par vos adaptateurs de droite

Les 2 prochains articles de cette série dédiée à l’architecture hexagonale parleront du sujet des health-checks (comment savoir si notre hexagone est en forme ou pas) mais aussi de la comparaison avec une variante du pattern, à savoir le Functional Core (imperative shell). A très vite.

Thomas

Don't be the fool looking the master's finger

More and more I hear people promoting either the Spotify model or the Google one. In a nutshell, they are saying: "if we copy them, we will be successfull" (regardless of their own culture or context). Also, some people think that you can change your own culture by simply adopting web giant practices. Remember Culture? It's that tiny thing that eats new strategies (and practices) at breakfast every morning ;-) Peter Ducker if you hear us...) This reasoning makes me think about a Chines proverb:

"When the wise man points at the moon, the fool looks at the finger"

Let me try to explain why. Spotify first:

The Spotify model

I won't detail that case too much and let you read Ben Linders' InfoQ post instead: Don't Copy the Spotify Model

My main point is that people should be inspired by the same driver and objective as Spotify (i.e. Aligned Autonomy) instead of copying and focusing on one strict organizational answer found by the Spotify guys during their Lean journey.

Here is the moon (i.e. the Aligned Autonomy objective):

And here is the finger our entire industry is focusing on instead ;-(

What bothers me more in that whole Spotify consultancy feast is the lack of result from an end-user perspective (on that topic, see my previous post: (controversy) Is Spotify a model to follow actually?)

The Google bias

Google is probably the case that annoys me the most. And I won't talk here about their problematic business model and its impact on our privacy (The client is the one who pay. If it's free, then it's because you are the product...Sad Panda)

No, I will just talk here about the huge bias most of us have when we talk about that successfull company. For many people, every post, practice, interview technique, tool,... coming from Google is took as gospel truth (regardless of everyone's context in term of funding, activity, culture, contraints, etc).

A little bit like with Spotify, most of us are looking at the master's finger, saying: "if we want to succeed, let's follow their advices and work like google does!" (last example here: Why do some developers at strong companies like Google consider Agile development to be nonsense?).

Before you follow that dangerous path, let me tell you a few things about Goggle. Or let me sum up it with a picture:

For more details, see the article: Google Makes So Much Money, It Never Had to Worry About Financial Discipline—Until Now.

A Cash-Cow story

Yes, Google has a tremendous a cash-cow (i.e. advertising) and is desperately trying to find its next cash-cow massively investing on whatever-is-cool-with-top-notch-engineers.

Then, unless you are rich like Donald Trump, don't try to mimic Google or to blindly follow their advices: almost all their projects are loosing money and work because they are founded by their advertisement cash-cow. If you don't want to be like the fools looking at the master's finger, don't listen people saying that you should work exactly as google does.

At best, you can be inspired by their striving for excellence (which is very impressive, mos def).

What they are currently building and lobbying with all our data and the destruction of our privacy (with concrete consequences on our lifes), will probably deserve another post...

Hexagonal != Layers

People around me know that I am a strong supporter of the hexagonal architecture (a.k.a. ports & adapters). Whether at work, during conferences or around coffee machines, I do not miss an opportunity to try to highlight the value of this architectural pattern created by Alistair Cockburn and which is DDD-friendly.

For me it's a real mystery: why almost no one actually leverages on hexagonal architecture, despite its huge ROI and value for projects...

During these discussions with people, a remark keeps coming again and again:

"... hexagonal and layers architectures are the same, right?"

Hell no! The aim of this article is to explain how different they are.

Some history about the "Layers" pattern

The concept of layers is not new in IT (eg OSI model), but the best description of this architecture pattern for me can be found within the POSA reference book (vol 1, published in 1996).

And like with any pattern, it is much more interesting to remember the intent / the problem to be solved than the solution. 

Otherwise there is a risk we use it because we know it instead of using it because it fulfills to our need...

I used to see many people using Layers by habit and without being able to justify why they use it. Let's make a test: ask colleagues who use it around you why they do it ;-)

Before comparing Layers with Hexagonal, let's resume the intent of the Layers architecture patterns: "helps to structure applications that can be decomposed into groups of subtasks in which each group of subtasks is at a particular level of abstraction."

The main objective: managing the complexity of a large application by decomposing it.

Now let's have a look at the solution: "Structure your system into an appropriate number of layers and place them on top of each other (...) observe the following rule: a J layer provides services used by its top Layer J+1 and delegates subtasks to its bottom layer J-1 (...) The main structural characteristic of the Layers pattern is that the services of Layer J are only used by Layer J+1; there are no further direct dependencies between layers."

The respect of the latter rule is crucial; it allows to limit the impact of the replacement of a layer with another one (only the top layer and possibly the bottom layer being impacted).

Although the pattern leaves open the number of layers, it is clear to me that the interest of the solution starts from 3 layers (for at least J,  J+1 and J-1). In fact in real life, you can usually count the number of layers within an application by knowing the number of tech leads previously involved on the project ;-) Leading to what we called a "Baklava architecture" (anti-pattern).

In the past I've worked on projects where there were so many layers... it was difficult to understand and follow the application / business logic.

I initially become interested in the Hexagonal Architecture to fix the poor signal / noise ratio found in those Layers architectures.

Hexagonal architecture: origin and usefulness

Created by Alistair Cockburn in the 2000s, the hexagonal architecture was designed to prevent the infiltration of the business logic into the UI code (such infiltration leading to less testable and more difficult to maintain apps).

The proposed solution is to divide our software in 2 distinct regions:

• the inside (i.e. the business application logic)
  and 
• the outside (i.e. the infrastructure code like the APIs, the SPIs, the databases, etc.).

2 distinct zones. No more, no less. 

Two distinct areas of our code with adapters positioned in what looks like a DMZ to protect the domain code from the infrastructure one. We just have to rely on the Dependency Inversion Principle (DIP) and the Repository pattern to prevent the domain code to be binded to the infrastructure code when it want to get some information outside, and voila!

Adapters? Ports?

There has been much misunderstandings and debates on this subject but after having check with Alistair, the "port" describes the intention (in C# or Java, a port is an interface belonging to your domain), whereas the "Adapter"is the code that bridges the 2 separated worlds: the infrastructure code where the adapter belongs and the Domain code the Adapter interacts with via a port.

Since I already wrote a previous post about the Hexagonal Architecture , post a code sample of the hexagonal architecture but also made a talk with cyriux (slides available here), I won't detail further here.

How the onion case complicates things

There is a variant of the hexagonal architecture: the onion architecture from Jeffrey Palermo.

To be honest, I don't really like this version which is mixing the concept of layers (like an onion) with the outsourcing of the infrastructure code as explained by Alistair.

As people usually only remember the "Layers" part, I think Jeffrey's pattern is misleading because it lead people to miss the most interesting part of the hexagonal pattern IMO (i.e. the segregation in 2 zones which simplifies everything).

Still not convinced that Hexagonal is not the same as Layers?

For sure you can mix both patterns by putting layers here and there within each of the 2 areas of the hexagonal architecture (like Jeffrey did), but I don't think you should. Indeed, this complicates the architecture of your system and has to be considered carefully (why would we do that? does it bring value in our context?).

Such mix-in would be our hybrid choice, and is not part of Alistair's description of the hexagonal pattern.

Can we say that 2 patterns with different motivations and different solutions are equivalent? 

Not in my opinion. What bothers me most with this speech is that it prevents people to discover the real power of the Hexagonal Architecture ("yeah, I don't see why I should look in detail this pattern, cause it's already what we do with our layers..."). Sad Panda.

According to me, the hexagonal architecture has two main virtues:

1. To prevent domain code to be contaminated by the infrastructure code (DDD friendly)
2. To simplify the architecture of our systems (avoiding the accidental complexity associated with extra layers for no explicit reason)

Fostering simplicity is hard but valuable. 

Hexagonal Architecture helps us to simplify our applications' architecture (and thus maintenance) by dividing it in 2 zones instead of a baklava architecture.

I like to finish that post with a tribute to Matteo Vaccari ASCII art comparison between Layers and Hexagonal. These diagrams helped me many times to explain those differences to colleagues at work.

Enough diagrams, let's see some code now!

I've published a few C# code samples of hexagonal architecture on my github space:

1. On the (LunchBox) SimpleOrderRouting project as explained here
2. In the CQRS (without ES) app sample published here (with its hexagon composition made here or like this in the acceptance tests)

(controversy) Is Spotify a model to follow actually?

Spotify is a major flagship for the "Agile consulting industry" (the one that sells certifications ;-) ... but I have to say that the music-addict I am has been deeply disappointed by the Spotify offer. Used to be a Spotify customer for many years, last year I decided to quit Spotify to join another platform (i.e. Deezer). 

I don't know if there is a link or not with the way they work and try to scale agile (tribes, guilds, etc), but I found Spotify's ergonomy very poor as an end-user. More annoying: the lack of common reflex between the various Spotify apps (on mac, on windows, on iOS, etc), and the poorness of the Spotify recos. In more than 5 years of Spotify usage, I have almost never discovered a new artist/band based on my actual playlists and preferences (contrary to what I've discovered thanks to Deezer's recommendations).


As a software developer, that makes me wonder many questions about the relevance of the Spotify agile model in order to deliver value to its end-users/customers.

Event Storming: my rookie mistakes

This week, I was happy to be asked by the teams to organize a second Event Storming session after our first successful experience last week (details posted in my previous post here)

Motivated but still a rookie in that field, I asked my mate Radwane (radwane_h) to help me to co-animate this new session. Indeed, we planned to experience more advanced event storming concepts like Commands, Actors, Aggregates etc. and I was happy to leverage on his Event Storming experience for such journey.

This session didn't went as I expected and I thought it could be helpful -for any new Event Storming organizer- to share here few lessons learned.

"Where is our wall?!?"

It started with a first difficulty: the audience was not prepared at all to face another empty wall... Indeed, since we don't have any war-room for the project (but the ability to use nice meeting rooms here and there), I had to trash the previous tablecloth after our first session. This seemed logical to me. After all, and as I repeated many times to the audience the week before: "Event storming is not about producing a new model/deliverable. It's mostly a live brainstorming to efficiently distill business flow among various actors, to identify key concepts & get rid of ambiguities." (all of this being useful to impact our code & other official artifacts in a second step).

Yeah, this sound logical to me. Indeed, most of the posts on Event Storming you can find on the internet advocate for restarting from scratch every time. It allows you to master the flow, but also to let the door open for better approaches (the first "solution" being not always the best option). This may require more effort during the first round(s), but this is a way to prepare our mental models to save time later.

As a consequence I intended this friday to make us zoom on a central part we identified the week before, involving 2 bounded contexts and few ambiguities in between (at least in my mind ;-). But facing the hostility of some of the actors (harms crossed, frowned eyebrows): I didn't succeed to motivate them to rework on the same topic without our previous wall with stickies on it as a support. One said to me: "If we don't start with the previous wall. We are loosing our time here!". Of course I had the pictures of the previous wall on my phone, but I was afraid to turn it into a lonely exercise if I had to transcript them to the wall (with potential disengagement of the audience).

Even now, I'm deeply convinced that it would have take us less than 10 or 15 minutes to put back all the stickies for this subpart of our business flow and I still blame myself for not having convinced all of them to do it.

But the audience was not ready for such reboot. 

It was like if I made them loosing their precious time (I think our Event Storming session came after few days of other different workshops without precise goals or obvious results; thus the impatience and the fidgets).

Anyway. Facing the astonishment of 12 people in front of me (and an empty wall ;-), I finally asked them which topic they wanted to choose for the hour and a half to come. After few seconds of awkward silence and minutes of collective hesitation... we picked a topic that we didn't studied at all during the first round. Something that I will refer here as the "Onboarding" (I can't say more here for non-disclosure reasons). That was interesting, but as we event stormed I detected an overall disengagement from the audience (around 12 people). Excepting 2 very active contributors, the rest of the audience was stood up too far from the wall with stickies, behind a table that can't be moved. They were trying to contribute to the debate but were somehow helpless and disengaged more and more.

I realized -too late- that we picked a topic with almost no domain expert for it in the room... Sad Panda.

In those conditions, the benevolent help of my friend Radwane was a kind of waste (I was sorry for him). He helped me and the audience, but that was still an Event Storming without domain experts in the room... (for this topic I mean).

The only positive outcome here was that we all realized we knew almost nothing about this "Onboarding" process (important pre-requisite for the rest of the project). We also identified some domain experts to be interviewed on the business-side, and found other stakeholders related to this process that we put on yellow stickies (handful for the months to come).

half of our wall: we discovered few things anyway

Nonetheless, I was truly disappointed having detected annoyance from some domain experts. I felt that I've lost an opportunity to make us work on the core topic for the weeks to come.

I would wrap this debriefing by few observations I've made to myself last friday:

1. Even if I'm not able to get back with the gigantic tablecloth of the previous session, I'll bring some prints of the pictures I took on my phone. This will allow us to quickly and collectively reboot our session even with people reluctant to redo the whole thing from scratch (it's sometimes hard to think out of the 'efficiency' box)
   
2. Before starting to Event Storm with 12 people, I think I'll check first if we have at least one domain expert in the room for this topic... ;-)

Hope this help.

Event storming: Domain distillation under steroids

Last friday, I organized an event storming session at work (one of the major investment bank in France). It was my first time as an organizer and I must admit that I've been truly impressed by the efficiency of this way to distill a domain.

Created by Alberto BRANDOLINI, Event storming is a fast way to explore a business domain or problem with many people in the room -including non-IT stakeholders. In a nutshell: you put every domain experts in the same room with few ITs and you make them describing what happen or must happen in their Business. To do so, you make everyone naming and positioning domain events (stickies here) on a large time-line-oriented-paper on the wall.

"What do you mean by 'Domain Event'?" is probably the first question you'll face as an event storming organizer. To quote Mathias VERRAES (another DDD expert and event storming practitioner), a domain event is

"Something that has happened in the past that is of interest to the business"

We are NOT talking about IT details here, like applications, databases or any button click... (unless its part of your core domain like for Google or other ad-resellers). That's why I stated at the very beginning of our session: "Let's forbid us to speak IT or technical things for the couple of hours to come. Business only!".

"It's developer's understanding, not your knowledge that becomes software!"

"What will be the outcome of our session?" is another legitimate question from the audience when it's the first time. In our case, the objectives were:

1.  To share a common understanding of core business concepts and what to do within our project.
   
   The project I'm talking about is an ambitious greenfield IS reconstruction for an entire business line. The fact that we have to deal with all functions -front to back- is a real challenge. Indeed, there are many bounded contexts and teams involved in our case. Everyone coming from various functional and technical backgrounds. As a Domain-Driven-Design (DDD) practitioner, I was recently worried by the fact that we were talking a lot about technical data formats (mediums/vehicules) or ambiguous concepts during workshops, and less about the core business concepts in stake. Fortunately we have lots of expertise here, tons of talented BAs & IT specialists. But to quote Alberto when he faces recalcitrant domain experts: "It's developer's understanding, not your knowledge that becomes software". In other words: it was time for non-domain-experts like me of few other developers to catch-up with the others.
   
   
2. To Fight vagueness and to clarify various ambiguous concepts faced during our previous workshops. The outcome here would be to initiate a 'lexicon' for our project, taking care about the scope of the definitions (by indicating the bounded context to which a definition applies for instance).
   
   For non DDD practitioners: a Bounded Context is a context within which a model applies (e.g.: marketing, negotiation, settlement, etc.).  Although it is important for a model & language to be consistent within a given context, we can perfectly have the same word indicating completely different concepts - or most likely - different perspectives on the same core concept depending on the context or usage we are talking about. Working without those contexts in mind could be really misleading. Moreover, sharing the language of our business (a per bounded context exercise) is also the starting point to have what we call in DDD an Ubiquitous Language (i.e.: to always rely on the language of the business -including within our code). As benefits, we avoid misunderstanding and "translation" issues, but we are also able to have efficient discussions with the business all along.

By the way, I forgot to tell you that there were no people from the business within our event storming. We were 11 people in the room including tons of domain experts in various contexts; profiles such as Business Analysts (mainly), domain-expert-IT-managers, and some developers/tech leads & a use-case-driven architect ;-)

Modus operandi

First thing's first: the preparation. Discovered in 2013 during a DDD meetup in Paris with Mathias VERRAS, I've since read and watch lot of things about event storming. Nonetheless, the fact that I was quite noob in this field made me asked tons of questions before our real session at work. And I'd like here to thank my friend Tomasz JASKULA (co-organizer of the Paris DDD meetup) but also Mathias, Alberto and Emilien PECOUL for their time and availability when I asked questions on Twitter few hours before my session.

Our event storming was schedule between 2PM and 4PM this Friday. In term of logistic I finally decided to go to the mall aside at lunch, in order to buy a roll of garden tablecloth to make what Alberto calls the "unlimited modeling space" on the wall.

Yes, Roberto warn me few hours before on Twitter: "make sure that location, stickies availability and space aren't constraining the discussion." So I also came with a bag full of working markers and tons of stickies.

source: Event Storming recipes (A.BRANDOLINI)

Before pinning my tablecloth on the wall, I moved out all the chairs in a dead-end corner of the room so that:

• people remain active during the workshop
  
  and
• to be able to observe the body language of all the attendees (much more complicated when they are seated as stated by Alberto).

Indeed. Like when we are playing poker (I confess ;-), it's very important to observe low signals, body languages and tells that betray possible disagreements among people (whether it's due to introversion or political agendas). As said earlier on the phone by Tomasz, "one of the objective of event storming is to identify possible disagreements on how we understand our domain". Either to settle them on the fly, or to park them for after (during another session for instance).

Once the room ready, people started to join me. It took me less than 8 minutes to explain the rules and the objective of this event storming to the eleven attendees before we put our first stickies on the wall.

There are many kind of stickies and possible approaches for an event storming (I let you read the "Event stormers" google plus group or Alberto and Mathias' blogs for further information). But this friday, we focused on "Domain Events" only (not the Commands, the Aggregates or the Actors). Thus, we almost only use "orange" stickies with past-tense sentence on it (e.g.: "Price Requested", "Order Executed", etc.).

source: Event Storming recipes (A.BRANDOLINI)

After one hour and a half, we wrapped up our session by identifying all the core concept of our domains (reading what we wrote on our stickies to do so), and started to collectively gave definitions for few of them on the left side of the tablecloth.

Takeaways from this first experience

• Used to Reactive Programming or CQRS architectures since various years from now I am pretty comfortable with the notion of event ;-) But I was pretty impressed that everybody followed the "Use past-tense sentence" rule (on Domain Events stickies). It sounded really a natural way for everybody to tell our business story.
• It seemed important to clarify from the beginning that we weren't here to produce another new official model/artifact for the project but to interact instead. Indeed, we were here to be on the same page and to collectively explore ambiguous part of our Domain. I had the feeling that it helped some of us to relax and not to fall into model perfectionism.
• It was hard for some of the attendees to avoid talking about IT things, applications, existing systems (even from Business Analysts). Nonetheless, we collectively succeeded to stick to our initial objective ("Business only!")
• Disagreements about some core concepts (e.g.: trade done vs. order executed) were quickly resolved with the support of all participants in the room able to add views and precisions (including naive questions from the non-experts). Visually also, with the support of other related Domain Events all around.
• It's been hard for some of us to realize that things that are currently done by existing IT systems correspond to real (Business) Domain Events (e.g.: the trading portfolio routing/attribution)
• Some people (4/11) started to take back some chairs and to sit. I hesitated to ask them to stand up like the others but after few minutes of observation, I realized that they were still actively contributing, answering to questions from the others. At the end of the day, I had the feeling that they needed somehow this background position to be able to see the big picture on the wall.
• We had few sub-groups during the session (distilling a subpart of the wall in parallel during few minutes here and there), but almost everyone actively followed the overall business story. Before the session, I feared some kind of post-it contention on the wall with 11 people (and disengagement as consequence) but nothing like that happened. Event Storming is really a fluent way of working together. It's amazing the efficient bandwidth we had to discuss and get in-sync with multiple people.
  
  
• As stated by Emilien PECOUL on Twitter, it is important to let people do their things and to not constrain them too much: "Let paralell discussion emerged". Nonetheless, I found interesting to make few "attention call to everyone" when we had to decide an important point or to answer a key question.
• To quote Alberto BRANDOLINI: "it is KEY that location, stickies availability and space aren't constraining the discussion". After this first Event Storming session at work, I can't agree more. The space or lack of space is really impacting the discussion. BTW during the session, I had to pin another piece of paper on the right while everyone was busy on my left, dangerously close to the end of the tablecloth ;-)
• Fortunately, I'd read Mathias' "Facilitating Event Storming" blog post before organizing this session: "Know when to step back. Don’t do the modelling, guide the modelling. Ask questions". It allowed me not to intervene too much in the modelling as the organizer of this Event Storming session. In a nutshell, I just tried to ask questions, to rephrase or to identify blur zones or implicits concepts.
• As for any other workshop, it was crucial to ensure a benevolent atmosphere so that everyone feels comfortable and willing to contribute in public. We joked here and there during this 2 hours sessions, but it was really a positive co-construction mindset.
• I waited almost one hour before suggesting the group to trace bounded contexts on the wall (between the stickies). In my opinion: if done too early, there is a risk to constrain the space and thus, the discussion.

To conclude on this first Event Storming experience at work, I would say that I've been truly impressed by the efficiency of this dispositive.

I would recommend it for any business domain exploration

For this first experience at work we focused only on the Domain Events (i.e. the "Orange" stickies). It probably eased the ramp-up for everyone in the room. But I'm very keen now to retry an Event Storming with the other tools such as querying the Domain Events causality (i.e. identifying Commands, People or External Systems), but also by identifying clearly the Aggregates involved (Stickies with other colors).

But as I said on Twitter just after this session:

Stickies are just a pretext for high bandwidth discussions & ultra efficient domain distillation.

And for that, Event Storming is an awesome tool.