A zoom on the hexagonal/clean/onion architecture

Several weeks from now, I've conducted an important study that was related to the middlewares we were using in the entire pre-trade perimeter of the bank I'm working for. One of the analysis criteria was the dependency & the coupling towards the used messaging technologies for every application. Even if there were lots of legacy applications involved in this study, I was still surprised to see how strong the coupling was and how far the various messaging technologies were spread within a huge number of applications (with messaging data structures present in the core business layers in the worst cases). The consequence of that?!? Some very costly (and risky) situations for those applications when we need to replace a former messaging technology they were using, by a better (and less risky) one.

This is where I realized that It was the time for me to communicate more on the strengths and the benefits of the hexagonal architecture, which I also recently successfully foster for another project (just before the middleware study).

Apart from being a new buzzword, what's the hexagonal architecture concretely?

Also known as 'Ports and Adapters', or clean, or onion architecture (other variants I won't detail here), the hexagonal architecture is an applicative-architecture-style that helps us to focus on our business goals without being tied or jeopardized by our technical frameworks or infrastructure choices. 

In that model there is no such thing as 'front-end' (users interactions) or 'back-end' (db) anymore, but two primary areas instead: the inside (with applicative-use-case-handlers and business domain code) and the outside (with all our infrastructure code: db access, messaging & communication bindings, etc). If you now combine this model with the dependency inversion principle which states that High-level modules should not depend on low-level modules. Both should depend on abstractions, you can easily infer that this model dictates that you can only point inwards the hexagon / circle (infrastructure being the low-level stuffs in that context).

Interactions between those two areas (in and out) are achieved by ports and adapters (P/A in the diagram below). In a nutshell, events or clients requests arrive from the outside world at a port (i.e. a plug for a technology), and the technology-specific adapter converts it into a usable procedure call or message and passes it to the application layer.

Ok, it's all about focusing on the real topics after all, right?

Indeed, an important point with the hexagonal architecture is that we put all our frameworks, drivers and infrastructure related code to the periphery of your system. Because those should be only details for our applications. Nothing more!

Strengths and benefits of the hexagonal architecture (to impress your world during geeks meetings)

• Sustainability / Timelessness: by decoupling our application-business code from the tools we are using (i.e. the libraries and frameworks), we make it less vulnerable to the erosion of time and IT fads
• Testability: The usage of ports and adapters to communicate with all our infrastructure (e.g. db, messaging systems, etc) eases the usage of mocks in order to test our applicative services and domain code. Tests could even be written for our application service layer before we decide which technology to be plugged with its corresponding port/adapter (whether REST, SOAP, specific messaging, db, etc)
• Adaptability / Time to market: adding a new way to interact with your application is very easy: you just add a new port/adapter to support this new technology and that's it! You can usually have multiple ways or technologies to interact with your application
• Understandability: Rather than having a solution where use cases are completely lost or mixed within all the technical stuff, this architecture style states the emergence of an applicative-use-case-layer (with all your use case handlers in a dedicated module). The proper location to make scream our functional intentions
• Use case driven & DDD compliance:  Indeed, with this architecture style, we design our applications with our use cases in mind; not the number of persistence technologies or binding types we will need to support! A typical project may start without deciding what kind of database it needs for his persistency, and pick the proper technology based on real data manipulated and usages discovered after several iterations (what we've done in a project recently; even if it took me some efforts to explain to the project manager that I was suggesting that strategy on purpose, and not by lazyness ;-) Cause you remember? a pragmatic architect usually defer decisions about the choice of frameworks or tools to be used.

OMG: is the hexagonal architecture a silver bullet?!?

Calm down. Please... This is not the Xmas architecture, and there is still no such things as Silver bullet ;-) But as Vaughn Vernon says in his Implementing Domain Driven Design book (iDDD) : 

'The hexagonal architecture forms the strong fundation for supporting any and all those additional architectural options' (i.e. SOA, REST, event-driven, event-sourcing, CQRS, etc)

I don't understand a word of this post. May I read other stuff on this topic in real & solid english?

Sure. Even if it's not very gentle for me... Anyway, if you want to dig a little bit on that matter, I highly recommend you the reading (of at least the chapter 4) of the iDDD book. You can also read the original pattern description by Alistair Cockburn, or the more recent explanations by Robert C Martin about the clean architecture (here as post, or here in this fabulous video session). By the way, I have no doubt that the next book of Uncle Bob (after Clean Code & The Clean Coder) will be something like: The Clean Architecture.

Enough said: time to try it in action in your own projects. You won't regret it!

Metrics, Metrics everywhere...

... is a truly awesome presentation by Coda Hale. Even if its content is really interesting (in a nutshell: improve your decisions and time to market by measuring your code when it runs), I really like the format of his presentation. 

Very funny, fluent, and built like an Anaphora (repetition, repetition, repetition), his presentation is a real 'model of genre' in term of efficiency to spread a message. 

Available on youtube, Coda's presentation lasts only 30 minutes long (there are 15 minutes of Q & A at the end); it definitely worth the look:

http://www.youtube.com/watch?v=czes-oa0yik

'As developers we have a mental model of what our code does ... we spend so much time inside our heads, it's very easy to mistake what's inside of our heads for reality (i.e. to mistake the map for the territory)... we can't know until we measure it'

+1

(thanks to my mate Cyrille for this resource)

The technology behind an equity trade

I highly recommend you to watch the excellent presentation of 'The technology behind an equity trade' made by John O'Hara during last QCon London 2013.

Clear as crystal and very informative, John is giving a good introduction of main bank functional blocks, before explaining the challenges every bank will have to face in the upcoming years (starting now ;-)

Finally, he ends explaining why technology will be so crucial in that industrial (and digital) transformation. Very nice session!

The reactive manifesto

Working in financial front-office since several years from now, I'm used to build reactive systems. 

In a nutshell, they are systems that are ready to respond to stimuli (whether market data updates, client requests for prices, spreads modifications, position changes, etc), in order to react as quick as possible (in milliseconds or less), in a world where you must constantly protect you against waves or throughput peaks of data coming into your systems.

To achieve that, we mostly build systems that are event-driven, stream-oriented, scalable (vertically, but also horizontally), performant (with conflation enabled, GC-less strategies, ...), and resilient (i.e. recovery-oriented & production ready).

Today, one of my mate showed me this new reactive manifesto which is all about that kind of systems we used to build. I found it very readable and informative. Check this out by yourself:

http://www.reactivemanifesto.org 

And welcome to our reactive world!

Understandability Driven Development

Thought you knew every kind of possible ...DD?

You should probably read the excellent white paper explaining Raft: the new clustering distributed machines consensus algorithm recently invented in Stanford to add a new one (UDD ;-)

Except a few things, their article is very clear and informative. But what I really like more than everything else is their approach. Indeed, they elaborate their algorithm with understandability as a core driver. The idea was to make possible for a large audience, to understand their algorithm comfortably.

Always interested by the challenge of how to deliver durable solutions and even if we are talking about an algorithm here (and not implementation), this is the first time I'm reading someone explaining that understandability is not a nice to have ... but a must have.

Pretty cool!

Update (July, 26th 2013):

• About the ‘why’: https://ramcloud.stanford.edu/wiki/download/attachments/3703006/SEDCL_forum_2013_raft.pdf
• About the ‘how’: http://raftuserstudy.s3-website-us-west-1.amazonaws.com/study/raft.pdf

The pragmatic architect

is obviously use-case-driven

A pragmatic architect is someone that gathers use cases and constraints before thinking solution.

Even more, a pragmatic architect cares about the 'why are we doing this?' (i.e. the underlying business goal, that is too often not clearly explicited and shared). He reviews and confronts any option or action with this initial business objective. He tries to deliver projects that will have impact, not only to ship software.

A pragmatic architect does not confuse architecture with tools and frameworks. He also likes architectures to scream their intentions.

knows there is no such thing as 'absolute truth or architecture'

A pragmatic architect knows that there is no absolute truth or solution. She understands that every choice or trade-off is depending on a context (including human considerations).

A pragmatic architect experiences what she recommends to the others. She stands by her choices with the rest of the team or stakeholders.

A pragmatic architect fosters situations where people can live without her. She tries to setup systems that make people fall into the pit of success.

is also a pragmatic programmer

A pragmatic architect is also a pragmatic programmer and a software craftsman. He really enjoys coding and to improve himself at it. If he could not code during working hours (i.e. the slides and diagrams kingdom), he may contribute to open source projects at home.

As every pragmatic programmer, he likes to script and automate every recurrent and painful things as much as possible. He is obviously a big fan of Continuous Delivery.

A pragmatic architect also understands that Continuous Build is not Continuous Integration.

A pragmatic architect is not only talking about Test Driven Development to every reluctant mate, he tries to show the benefits of TDD as often as possible during concrete pair programming sessions.

A pragmatic architect knows the 9 indispensable rules of debugging, and is always willing to remind them to anyone that may need it. A pragmatic architect is also an ego-less programmer.

A pragmatic architect is capable to discuss the strengths and weakness of Object Oriented Programming or Functional Programming to other mates. He likes to step aside and get perspectives on things, and fundamentally needs to understand the reasons of every choices.

A pragmatic architect knows about patterns and anti-patterns, he values GRASP, GOF and POSA patterns, but also cares about the Capacity and Stability ones. As every software craftsman, he also embraces the SOLID principles. But more than anything else, he likes that things stay as simple as possible.

A pragmatic architect likes abstractions, but is aware of the law of leaky abstractions. He also knows that "the pitfall of dogmatism is inherent in any effort of abstraction".

A pragmatic architect has lots of ideas, but he likes nothing more than to share and to debate them with others. But mostly to confront his ideas with the real world. Also, a pragmatic architect does not fear to try. He knows that failure is mother of success. And he usually does not put all his eggs in one basket.

A pragmatic architect does not fear risks, he rather mitigates them.

cares to understand the business stakes.

A pragmatic architect is curious, this is one of the reasons why she chosen this job. And even if she's an hardcore technical addict, a pragmatic architect is someone that cares to understand the business stakes.

A pragmatic architect always prioritizes the work according to the value it brings to the project. This is why she usually defers decisions about the choice of frameworks or tools to be used (db for instance)..

A pragmatic architect always asks 'Why?' five times, so that she can find out the real root causes, intentions, or needs behind her interlocutors words.

A pragmatic architect likes to explicit every kind of implicit. She makes everyone to be on the same page. She's even able to ask naive and silly questions she already knew the answer, just for shy people in the room to heard it. She usually ask people to re-phrase things in order to check they have understood what was said.

A pragmatic architect has not only a vision for the project or the IS, but she cares to share it with ALL the stakeholders involved.

A pragmatic architect is an enthusiastic team player. She really enjoys to collaborate with people and teams.

A pragmatic architect understands that Behavior Driven Design (BDD) is not a matter of tools, but a matter of collaboration and communication among various actors (Biz/BA, ops, QA & DEV).

A pragmatic architect is agile, but doesn't feel the need to repeat it to everyone. She is now more concerned about how to scale agility within the organizations, and how to improve current practices.

A pragmatic architect is as capable of putting project's primary focus on the core domain with Domain Driven Design (DDD), that understanding the technical challenges of all technical stacks. She is multilingual (biz, managers, dev, ops,...).

A pragmatic architect understands that we should stop trying to model the real-world, but only what matter to our software usages instead.

A pragmatic architect understands that event sourcing is not only a technique for technical challenges like High Availability, but something that may bring real business added values.

does not guess, he measures.

A pragmatic architect doesn't like early optimizations. A pragmatic architect does not guess, he measures instead.

A pragmatic architect has both network and hardware skills. He knows not only HTTP, but he masters IP, TCP, UDP... He also tries to avoid having too many war stories, by learning from others' experiences. He also knows how multicast is working under the hood, and other stuffs like it's possible to do unicast with UDP, what is false sharing, what every programmer should know about memory,  etc. A pragmatic architect is able to understand the impacts of route flapping and network routing convergence on the systems we build. He is a real interlocutor for infrastructure guys and network engineers. He can do that because he is dramatically curious, and always willing to discover new things when he heard about it.

A pragmatic architect is a multi-threading expert, and this is why he tries to help people not to play with threads. A pragmatic architect also likes to spot the deadlocks at home, but hates to do that in production.

A pragmatic architect knows about Recovery Oriented Computing (ROC) and builds production ready systems. He tries to minimize the impacts of bugs on the system when they happen, rather than trusting the idea of a world without bug (ha, ha, ha ;-). Thus, he cares to setup some 'what if ...' workshop sessions before going under production, and to deal with exceptions at system integration points within the system he builds.

A pragmatic architect also knows and cares about mechanical sympathy. He also knows -thanks to the CAP theorem- that you can't have your cake (HA) and eat it (Utter Consistency) too! 

A pragmatic architect is still wary of benchmarks. He knows that almost all of them are done under conditions that have no relationship with what's interesting in the real-world.

A pragmatic architect remembers Schrödinger's cat when he profiles systems (and is also wary of heisenbugs).

likes 'easy in, easy out' technologies 

A pragmatic architect hates golden hammers. She constantly tries to avoid having her own ones.

A pragmatic architect likes 'easy in, easy out' technologies. That's kind of logical, cause she naturally does not like strong dependencies and coupling. And this is why she often promotes the hexagonal architecture.

A pragmatic architect is usually wary of frameworks and rather uses libraries. She tries to know when and where the abstractions will leak.

A pragmatic architect has more to propose than layers architecture only. When she proposes a layers architecture, she intentionally knows why, and is able to explain it to her mates.

A pragmatic architect knows various technologies such as LDAP, NTP, RDBMS, NoSQL, etc. She's really curious, and always willing to discover new things or paradigms.

Of course, a pragmatic architect likes, but also animates Brown Bag Lunches!

reads tons of books, but doesn't blindly apply things 'by the book'

A pragmatic architect is open-minded, and curious. He likes to break silos, and thus enjoys reading things on various other disciplines (not only IT).

A pragmatic architect enjoys to step back and raises his head; he's usually good for the job thanks to his systemic mindset.

A pragmatic architect reads tons of books, but he does not apply things 'by the book'.

A pragmatic architect attends to various IT conferences or 'geek' meetups. He's very social, and always willing to share what he has learned to the mates that couldn't attend those events. He then  spreads the words via twitter, blogs, mails, presentations, coffee breaks, etc.

A pragmatic architect may use acronyms (when he's tired ;-) ; but is always willing to explain their meaning when he sees question tags in the eyes of his interlocutors. He cares about how people receive and understand his words and messages. Thus, he's able to use tricks to re-focus his interlocutors on the core objectives when they diverge.

I already told you, but a pragmatic architect likes simplicity.

A pragmatic architect is aware of the power of humor to communicate with others (except within this current post ;-P

has no IT religion

A pragmatic architect has no IT religion and is not dogmatic. He doesn't care stupid debates like java vs. NET, etc. He's able to think out of the box, and adapts to contexts and people that will implement the systems.

A pragmatic architect cares about others point of views.

A pragmatic architect is able to convince others with a crystal-clear speech. 

But above all, a Pragmatic architect is able to adapt his positions and to challenge himself. 

Last update: November 16th 2013

The 10 commandments of Egoless Programming

Thanks to my friend dupdob, I've just read the excellent post on kitchensoap's blog: on being a senior engineer.

This must-read-post includes the excellent 10 commandments of Egoless Programming, apparently found on @wyattdanger's blog post (on receiving advice from his dad). Simply Awesome...

1. Understand and accept that you will make mistakes. The point is to find them early, before they make it into production. Fortunately, except for the few of us developing rocket guidance software at JPL, mistakes are rarely fatal in our industry. We can, and should, learn, laugh, and move on.
2. You are not your code. Remember that the entire point of a review is to find problems, and problems will be found. Don’t take it personally when one is uncovered.
3. No matter how much “karate” you know, someone else will always know more. Such an individual can teach you some new moves if you ask. Seek and accept input from others, especially when you think it’s not needed.
4. Don’t rewrite code without consultation. There’s a fine line between “fixing code” and “rewriting code.” Know the difference, and pursue stylistic changes within the framework of a code review, not as a lone enforcer.
5. Treat people who know less than you with respect, deference, and patience. Non-technical people who deal with developers on a regular basis almost universally hold the opinion that we are prima donnas at best and crybabies at worst. Don’t reinforce this stereotype with anger and impatience.
6. The only constant in the world is change. Be open to it and accept it with a smile. Look at each change to your requirements, platform, or tool as a new challenge, rather than some serious inconvenience to be fought.
7. The only true authority stems from knowledge, not from position. Knowledge engenders authority, and authority engenders respect – so if you want respect in an egoless environment, cultivate knowledge.
8. Fight for what you believe, but gracefully accept defeat. Understand that sometimes your ideas will be overruled. Even if you are right, don’t take revenge or say “I told you so.” Never make your dearly departed idea a martyr or rallying cry.
9. Don’t be “the coder in the corner.” Don’t be the person in the dark office emerging only for soda. The coder in the corner is out of sight, out of touch, and out of control. This person has no voice in an open, collaborative environment. Get involved in conversations, and be a participant in your office community.
10. Critique code instead of people – be kind to the coder, not to the code. As much as possible, make all of your comments positive and oriented to improving the code. Relate comments to local standards, program specs, increased performance, etc.

Domain Driven Design in a mindmap

I have to admit, when I first heard about DDD, I thought it was a simple buzz-word for guys that have read Fowler too much, and that try to sell lots of consultancy around a trendy concept. A kind of consultancy vaporware.

I was wrong. 

And I should have read Eric Evans' amazing blue book instead of reading lame web articles and posts on that topic, that led me to think that DDD was bullshit, another trendy acronym.

Following advices from my mates Cyrille and Alex, I've read the blue book several months from now. And I have to admit, this was a mindset revolution for me (same strength as when I discovered eXtreme Programming and agility).

DDD has dramatically changed the way I work 

(and the way my close mates work too). From now, I even turn myself into a DDD enthousiast, trying to promote the approach to whoever need to build use case driven softwares.

This is why I recently produced this mindmap, which help me to present the DDD approach to every team newcomer, in about an hour or two.

Hope this help.

Will you "spot the deadlock" with dupdop?

On "many cores", his blog dedicated to multithreading and concurrency issues, one of my friends proposed a new and very funny trivia game: "Spot the deadlock".

In a nutshell, dupdob will regularly present code snippets which exhibit some concurrency related problem, and will let every one of us finding what concurrency issue is present, why does it happend, and how can it be fixed.

The idea is to leave us one week to give our answers, before all the comments are shown.


The very first (easy) episode is already available here:

http://dupdob.wordpress.com/2013/01/23/can-you-spot-the-deadlock/

Will you spot the deadlock ?

Have fun!

Cloud computing, Economic intelligence & Patriot act

Recently discussing with some colleagues, I realized that very few of us know the risks of National Economic Intelligence related to the usage of public cloud solutions provided by US companies.

Indeed, the Patriot Act, enacted in 2001 in US, gives full powers to the United States in investigating. Thus, any U.S undertaking must provide "sensitive data" required by the Federal government (NSA for instance), and that, whatever the place where they are stored. 

Worst: all of that may be done without notifying it to the authorities of the country or the company that has been investigated.


My point here is not to make us fall in a fully paranoid mode. But I think it is important to have these risks in mind when we advise our companies to move towards any public cloud solution (mostly provided by american companies: Amazon, google, Microsoft, etc.).

Of course, public cloud solutions are still valuable for most of the companies and data. But for other companies with crucial and highly competitive advantages, it's important to think it twice before moving into such public cloud solutions...

In a nutshell: public cloud not for everyone, not for every kind of data.

Some posts or newspapers articles about it:

• Defense giant ditches Microsoft's cloud citing Patriot Act fears (zdnet)
• The Patriot Act and Your Data: Should You Ask Cloud Providers About Protection? (cio.com)
• Snowden link to encrypted email service closes

and in french:

• Patriot Act : vos données lui appartiennent (from my friend Stephane Bellec @01net.com)
• USA Patriot Act : un risque majeur pour la confidentialité des données dans le Cloud (lemonde.fr)

Last update: August 18th, 2013

wiked! the solution to DRY all our projects KM

Since my latest post: It's really time to DRY our apps' Knowledge Management! I had the occasion to share this topic with several mates, and thus to discover that the maven site plugin was perfectly fine for what I intended to do (thanks to Christophe LALLEMENT & Alexandre NAVARRO by the way).

Thus, no need to reinvent the wheel for that (which is perfect fine for me).

So I started to play with maven (quite new for the .NET expert I am), and especially with the maven site plugin ecosystem. At the end, it took me several hours in order to fully understand how to use it properly. With some questions like:

• Which maven plugins (and versions) should I configure within the pom.xml I intend to use in order to generate the web site for my project (which is not a java project by the way)?
• Which minimal pom.xml could I set in order to generate a web site with the mvn site command-line (important for non-java project, and to KISS)
• Where should I put images and other resources in order to be able to reference them properly from my markdown pages
• Why and when should I have to call `mvn clean` before the `mvn site` command-line in some rare cases (like when changing the project name whithin the pom.xml file)
• How to set a better look n feel to the overall generated web site?
• Which markdown syntax was working, and which one was not well supported by the tools (i.e. the atx-style headers)?
• etc.

I finally intended to package something that would help me (and others) to quickly setup such web site generation solution for every new project Knowledge Management.

Thus, i'm proud to present you the wiked! solution,  now available on gitHub.

More than a simple bootstrapper that will ease you to earn some time when setup-ing such a solution for your project, wiked! also offers you a structure and some content for all your projects web sites (see the values and practices section for instance).

And since the DRY principle was one of my driver, I won't fully describe here what wiked! is all about. If you are interested, you can have a look at the wiked! ReadME.md

Hope this help.

It's really time to DRY our apps' Knowledge Management!

While I was reading the amazing Pragmatic Programmer manifesto, several years ago from now, I really enjoyed the "Keep Knowledge in Plain Text" principle (tip 20).

Unfortunately, I never really took the time to apply this principle seriously for the Knowledge Management (KM) of the various projects on which I worked. Every time, I lazily put all the team/application related informations whether in a wiki, or as MS Office documents stored on a Sharepoint... (sad panda ;-)


Last time, while I was having a lunch with a colleague of mine (a well-known and passionate french craftsman), I realized how lazy I was so far regarding this KM topic. With 2 or 3 words only (read-only wiki, Markdown documentation, SCM), Cyrille starry-eyed the dormant craftsman in me.

Indeed, since:

• wiki web sites for DEV teams usually finishes badly (e.g.: lots of different wikis/sections for the same component/teams, no proper knowledge of what is still relevant and what is deprecated, structure mess, etc)
• documentation created separately from code or components is less likely to be correct and up to date
• it’s not easy to compare/diff MS Office documents
• it’s important for every piece of knowledge to have a single, unambiguous, authoritative representation within a system (DRY principle)
• the text documentation has becoming more important than ever since the advent of the blue book (to express the ubiquitous language of our projects, to clarify some large-scale design intents, etc)

It would be a good idea for us to find a way to get rid of traditional team wiki site, and instead to write every documentation in:

• text files written in Markdown format (we may benefit from any Markdown existing tool/editors)
• stored within the SCM of our project (e.g.: SVN, git)
• and to make our Software Factory (re)generate a static web site (like a read-only wiki) every time we changed those Markdown text files from within our SCM. 

As benefits, our team/application documentation will:

• always be up-to-date and versioned
• easily diff-able (text files with Markdown format)
• respect the dry principle (with our SCM as its golden source)
• easily browsable by everyone in a readonly but readable wiki-like web site (DEV, QA, Support teams, …)
• easily modifiable by team members in a well know and official location (as easy as creating/modifying a text file in a SCM)

Of course, the idea here would be not to reinvent the wheel, but to reuse any existing Markdown renderer & static web site generators instead (the step beyond the github Readme.md file). But if I can’t find something and make it out quickly, I would probably initiate an open source project to implement such system.

Cyrille was right: it's probably time for us now to kill wikis! and to reinvent them.

Last but not least: a very interesting post of Cyrille about Collaborative Artifacts as Code.



BREAKING NEWS: I found a solution for my use cases, described here.