Bloated Domain Objects And CQRS (Command, Query Responsibility Segregation)
 

Problem of Bloated Domain objects

In business software applications, the domain objects (entities) are used to represent the business domain. As the application grows and adds more business logic, the service layer, mappers and other patterns gets applied. Often this leads to domain object becomes bloated and the related components become huge & un-maintainable.

CQRS solves the common problem of having a bloated Domain objects. The domain objects get bloated largely because of bounded context. The series of contexts which makes developers think that a single domain object is sufficient to handle all the related things. For example, a large Invoice object for handling Invoice, Shipment and handling change of address for customer . But in reality, these contexts (invoicing, shipment and change) need not be related to same Invoice entity.

What is Command, Query Responsibility segregation (CQRS)?

In order to simplify the Domain objects, CQRS proposes to have two types of domain entities.

  • those serving the command (ordering/assertion services)  – For example, SaveCustomer, CreateInvoice, ShipProduct etc
  • those serving a Query (request) – examples include GetCustomerAddress, SearchCustomer etc

With this separation, the complexity (number of fields, methods) of entities used becomes simplified. And hence the Data mapper layers & the service layers becomes more simplified.

Where can I use CQRS?

  • Largely complex system: Applying CQRS on a simple CRUD operation based system is a over kill. When there is a domain heavy system, like banking and financing systems, LOB applications where business logic, lots of boundary conditions are heavy. Where it makes DDD (Domain driven design) provides high value.
  • Situations where you will apply Microservices, Eventual consistency and Event Sourcing. When we have separation of concerns using CQRS, the microservices becomes much simpler to design and maintain. With Event sourcing we are focused on getting the data (query) from other related sources and is what CQRS propagates.

Final words

CQRS is a carefully thought out pattern for simplifying large and complex systems. And it should not be applied for simple green field applications. Do let me know what you think.

Micro-Services, Eventual Consistency and Event sourcing patterns
 

Microservices is a really becoming a famous architectural pattern that most of the new software written these days, just apply them. One of the most important differentiation between the traditional web services and micro services pattern is the amount of common stuff across different subject areas. This calls for a discussion on how Eventual Consistency pattern is mandatory for successfully implementing micro services.

Micro-Service Pattern

Generally in a micro-service pattern, the API’s are split in to small subject areas. For example for a CRM application, the subject areas are

  • Customer information – like name, address, email, phone
  • Appointment information – which customer, sales person, when, where
  • Relationship management – sales/manager, what products, interests
  • Campaign data – offers, deals etc

Then micro-services are built for each of the subject areas.  The microservices are logically and physically separated from each other. ie there is no sharing (code, database, component etc) between any of these micro services of these subject areas. Pictorially its looks something like this.

Applying Eventual Consistency Pattern

In Micro-services, there is no data that is shared across the micro services. In order to synchronize the data across these isolated storages of these services, we need to apply the eventual consistency pattern. You can read more about applying the pattern correctly here. The simpler way we can achieve consistency across these micro-services is through Event Sourcing pattern.

Event Sourcing

Event sourcing is a process of capturing application state changes in the form of events.  An example of events are customer created, customer updated, Deal created, etc.  Other systems listen to these events and they take relevant action. You can read more about event sourcing here.

Conclusion

Event sourcing is the new way of storing changes to systems and help in making micro-services eventually consistent. These patterns together form well maintainable, reliable and scalable systems in the modern world.

BDD (Behavior Driven Development)
 

In the previous article we looked at how to do Test Driven Development (TDD) while doing the XP (Extreme Programming). In XP, the implementation of the tasks are done in the steering phase of the iteration planning. One of the tenets of the steering phase is “functional tests are run”. In this article we will see how to create functional tests using Behavior driven development (BDD) and its benefits.

Following are the some of the benefits of automated functional tests.

  • Functionality experienced by the User is tested
  • Saves testing time for the developers
  • Test in every environment (Dev, QA, Stage) without much effort
  • Generates confidence in every stake holder

Tools required for doing BDD in .Net

There are various tools available to do BDD in .Net. I have listed few of them below

SpecFlow is one of the most popular tools used in the .Net/C# community. And in this blog lets use the same. The tool comes as a Visual Studio Extension and can be downloaded directly from VS.

Creating a SpecFlow Project

Once SpecFlow extension is installed, the template required for doing BDD will be installed.

  • Create a simple “Class Library” Project in Visual studio
  • In the project, create a SpecFlow Feature file

Selecting the test framework

Before we Create/execute the scenario, we need to wire the SpecFlow with a compatible test framework. To do that we need to make changes to the app.Config file of the project.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />
  </configSections>
  <specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
    <unitTestProvider name="xUnit" />
  </specFlow>
</configuration>

In this example we are using XUnit as the testing framework. SpecFlow supports a series of testing frameworks and more details can be found here.

Creating the Behavior

Now, let us look at how to create a functional test using behavior driven approach. Let’s consider the following story while creating a POS (point of sale) application for a super market.

“As a Supermarket POS app developer,  i would like the API to Give the total amount while billing so that I can wire up API against the UI.

Let’s write the Behavior required in order to implement the above API requirement .

Feature: SuperMarketProductsAPI
	As a Supermarket Biller 
	i would like the API to
	Give the total amount while billing
@SmokeTest
Scenario: Find the total amount while billing
	Given Shopping cart is filled with all the items required by consumer
	And Campaign information for the month is available in the system
	When I pass the cart to create bill API
	Then the total bill amount should be calculated

The above specification is written using the custom language (testing DSL) create by the specflow.org. (Please read about what is DSL here.)

There are 2 parts to the above Specflow specification

  • Feature – Denotes bigger context (description/story) of the application
  • Scenario – Specific workflow or behavior of the system under the feature. Each scenario has the following sub-parts
    • Given, And – describes what we already assumed available
    • When – The specific action which will trigger the workflow/behavior
    • Then  – expected behavior

 

Creating step definitions

When we run the unit test corresponding to the above test we it will fail as there are no definitions corresponding to the above scenarios.

Now Right Click on the feature file and select “Generate step Definitions”

You will see the pop-up like below, select “Generate” and then “Save” the file.

The file will have the details about “what SpecFlow should do when test is executed”.

using System;
using TechTalk.SpecFlow;

namespace SuperMarketFunctionalTests
{
    [Binding]
    public class SuperMarketProductsAPISteps
    {
        [Given(@"Shopping cart is filled with all the items required by consumer")]
        public void GivenShoppingCartIsFilledWithAllTheItemsRequiredByConsumer()
        {
            ScenarioContext.Current.Pending();
        }
        
        [Given(@"Campaign information for the month is available in the system")]
        public void GivenCampaignInformationForTheMonthIsAvailableInTheSystem()
        {
            ScenarioContext.Current.Pending();
        }
        
        [When(@"I pass the cart to create bill API")]
        public void WhenIPassTheCartToCreateBillAPI()
        {
            ScenarioContext.Current.Pending();
        }
        
        [Then(@"the total bill amount should be calculated")]
        public void ThenTheTotalBillAmountShouldBeCalculated()
        {
            ScenarioContext.Current.Pending();
        }
    }
}

When you build the project and execute the test corresponding to this, it will fail. This is because none of the parts of the test (given, when, then) are having implementation.

Writing code to make it work

Now that the behavior has been created, we are good to validate that with the team and implement the code/functionality corresponding to the requirement. Of course using best coding practices like TDD 🙂

Making the behavior test pass

In order to make the behavior test pass we need to write the implementation in the “Step Definition” file.

namespace SuperMarketFunctionalTests
{
    using Newtonsoft.Json;
    using System.Collections.Generic;
    using System.Net.Http;
    using TechTalk.SpecFlow;
    using Xunit;

    [Binding]
    public class SuperMarketProductsAPISteps
    {
        Product P1 = new Product { Name = "test1", Cost = 5 };
        Product P2 = new Product { Name = "test2", Cost = 10 };
        ShoppingCart cart;
        List<Campaign> campaign;
        Bill bill;
        [Given(@"Shopping cart is filled with all the items required by consumer")]
        public void GivenShoppingCartIsFilledWithAllTheItemsRequiredByConsumer()
        {
            cart = new ShoppingCart { Products = new List<Product> { P1, P2 } };
        }
        
        [Given(@"Campaign information for the month is available in the system")]
        public void GivenCampaignInformationForTheMonthIsAvailableInTheSystem()
        {
            campaign = new List<Campaign> { new Campaign { product = P1, discount = 1 } };
        }
        
        [When(@"I pass the cart to create bill API")]
        public void WhenIPassTheCartToCreateBillAPI()
        {
            var client = new HttpClient();
            var response = client.PostAsync("http://myapi.supermarket.com", new StringContent(JsonConvert.SerializeObject(cart))).Result;
            var output = response.Content.ReadAsStringAsync().Result;
            bill = JsonConvert.DeserializeObject<Bill>(output);
        }
        
        [Then(@"the total bill amount should be calculated")]
        public void ThenTheTotalBillAmountShouldBeCalculated()
        {
            Assert.True(bill.totalAmount == 14);
        }
    }
}

Once the code is there we can see the behavior test pass.

Conclusion

BDD is very powerful way of ensuring high code quality in conjunction with business requirement. In Extreme programming (XP) , BDD is considered to be a very important practice where functional tests are mandated to run at least once during the iteration.  Is BDD/Functional tests helping your team/business in achieving success or not ? Do let us know your thoughts.

Choosing between Trunk based development(TBD) vs Feature branching
 

In a development environment like scrum, XP the branching strategy can significantly impact the overall speed of delivery of the product. There has been detailed documentation on the various branching strategies, and you can encounter TBD (Trunk based development), feature branching. In this article,  am covering the various factors that can impact the success or failure of a these branching strategy.

What is feature branching:

In this method, there is a code branch created for each and every feature. In agile based environments, typically, a branch is created for every story in the sprint.

 

What is Trunk based Development (TBD): When a single trunk/branch is used to create all the features of the product, then its called Trunk based development. Often a release branch is created when ever there is a release that is pushed to production.

Following are the various factors to look at before selecting Trunk based vs feature branching.

  • location of team members: when team members are co-located, the trunk is the best bet, helps us to get faster feedback. Also team can just talk about the code changes directly. Since the feedback will be faster in a single trunk, the team need to sync up more frequently. Co-location enables this sync trunk changes to happen frequently and faster. When there is a distributed environment (often in our teams), feature branching works best.

  • speed development environment: In a high speed dev environment, the trunk just shines as there are less and less process overhead, like branching, Pull request, review etc. Feature branching shines well when there is less speed in churning code.

  • size of code base: Insanely higher number of code lines can be better handled using feature branching. But for smaller code base, the cost of creating branches etc can be higher.

In our team meetings, we often talk about the right tools for the problem at hand. And using the right branching strategy is an important decision. We often question ourselves if the strategy is right for the given epic/sprint. If the answer is NO for few sprints, we know, we need to change something!

Please comment and let us know your opinion.

Maintaining IEnumerable-Yield Data Pipeline is Hard
 

C# and .Net had IEnumerable and yield since its early versions. And I think its one of the most misunderstood features of C#. This is because of the pattern of deferred/lazy execution implementation.  Before we delve deep into this let us understand the iterator model.

IEnumerable and IEnumerator:

This is the foundation of the C# Iterator pattern and there are a number of detailed articles about this by Eric Lippert. The simple example is here.

There is a basket full of Oranges.

 
public class Basket
{
    private readonly IEnumerable Oranges;

    public Basket(IEnumerable oranges)
    {
        Oranges = oranges;
    }

    public IEnumerable GetOranges()
    {
        return Oranges;
    }
}

public class Orange
{
    public int NumberOfSlices { get; set; }

    public void Peal()
    {
            
    }

    public void Consume()
    {
            
    }
}

And if we have to consume oranges, we will have to peel each one of them. So we can use a simple iterator pattern on them.

var basket = new Basket(new [] { new Orange(), new Orange(), new Orange() });
foreach (var orange in basket.GetOranges())
{
    orange.Peal();
    orange.Consume();
}

But the problem is, we need not consume all the oranges from the basket at one go (I am not Hungry ;-).
So how can I get one orange from the basket at a time? or whenever I am Hungry?

Yield:

C# Yield keyword provides an answer to this problem. Slightly changing the Basket class using Yield keyword. (signature of the GetOranges Method not changed)

public class Basket
{
    private readonly Orange[] _oranges;

    public Basket(Orange[] oranges)
    {
        _oranges = oranges;
    }

    public IEnumerable GetOranges()
    {
        for (int i = 0; i < _oranges.Length; i++)
        {
            yield return _oranges[i];
        }
    }
}

Now the basket will provide one Orange at a time. ie when MoveNext() Method of the IEnumerator is called. In other words, we have paused execution of the “for” loop in GetOranges method till the time the next Orange is required.

Or in more technical words, until you Enumerate, the data will not be fetched and once Enumerated, there is no data. It all boils down to “when you Enumerate”.

The Maintenance Problem:

This is tricky for developers who are inheriting an existing code base. A developer might accidentally add an Enumeration before the place where it actually has to. The next time or When the original Enumeration happens, it can not find any new data and the iterator just completes without any looping. This can happen very silently without any exceptions. So care to be taken to see where we are yielding and where we are Enumerating.

“Yield return” is a great feature in C#, but use it with caution.  Let me know your thoughts.

Perspective Designing
 

Recently, I was working with a colleague in refactoring one of our projects. As we added tests, we found few code issues and continued refactoring. Was feeling happy as our unit tests were rearing benefits. However, we know TDD or unit testing does not guarantee clean code. As we progressed, the naming conventions consumed a lot of our time. And eventually, it brought us to a discussion about why specific naming conventions can create a better design. Thought I will share our discussions and practices here.

While we design classes for application, we often think of it as a different subject than ourselves (programmer). When I say different subject, we think of it as a different object and not as a person. When a programmer considers classes/interfaces as personalities and thinks from the perspective of the class, design can change drastically. This is what we call “Perspective designing”. Let’s take an Example:

    public interface ITotalTaxCalculator
    {
        decimal Calculate(IEnumerable products);
    }

    public class TotalTaxCalculator : ITotalTaxCalculator
    {
        public decimal Calculate(IEnumerable products)
        {
            decimal total = 0.0;
            //add total of products etc....
            foreach (var product in products)
        	{
                using(var dbContext = new ProductContext())
                {
                    var productInDb = dbContext.FistOrDefault(prod => prod.Id == product.Id)
                    total += (total * productInDb.taxRate);
                }
        	}
            return total;
        }
    }

In the above example, the name of the class and interface are perfectly fine. But they are impersonal and it’s very hard to think of it as a person and bring in perspective thinking with these names. So we refactored them to ‘ICanCalculateTotalTax’ and  ‘TotalTaxMan’.

public interface ICanCalculateTotalTax
{
    decimal Calculate(IEnumerable products);
}

public class TotalTaxMan : ICanCalculatorTotalTax
{
    public decimal Calculate(IEnumerable products)
    {
        decimal total = 0.0;
        //add total of products etc....
        //blah blah blah..
        total += (total * taxRate);
        return total;
    }
}

These naming conversions have lots of inspiration from in NServiceBus for their class/Interface names. With the new class and interface names, it’s easy to think of them as personalities. However, this does not guarantee good design. So we needed refactoring. Perspective thinking comes in handy especially while we do refactoring When my colleague and I started putting ourselves in the place of each of the classes. We had very reasonable questions which triggered our object-oriented thinking.

Example1:  As ‘ICanCalculateTotalTax’ , why I am having database related behavior?

Example2: As ‘ICanCalculateTax’, why I am having logic to find which language it needs to be presented?

These questions helped us to refactor the code to follow good design principles. When we implement these interfaces/abstract classes, we have clarity on what the class is capable of doing. So we generalized these naming conventions & questioning attitude and derived below two rules to do Perspective designing (think like a class).

  • Give personality to the names of  classes/interfaces (example: ICanCalculateTax)
  • Use the Agile User Stories way of articulating what the class should and should not do. (example: As ‘ICanCalculateTax’, I should be able to provide behavior to calculate tax)

I think, “Perspective designing” can make classes more object-oriented and best practice like SOLID principles automatically fall in line. Let me know your thoughts.

SOLID Principles
 

Many of the old programmers have adopted SOLID principles and it is taught in lots of schools these days. Yet I find many people not clearly understanding SOLID principles. I think these principles are fundamental to creating good object oriented software and every developer should learn them. So I have series of posts for these principles

Single Responsibility : A piece of software always have a single responsibility!
What this basically translates into is that a piece of code should have one and only one functionality. The corollary to this would be that  a piece of code should have only a single reason to change. Some advantages of this would be to ensure minimal change to existing classes in case of a change in requirements, which avoids larger testing efforts for a larger number of classes changed, which cuts down on testing efforts and increase turnaround times, which leads to lower overall cost of ownership for the code, which saves $$.

Take the example of a logging class. As a software developer, chances are pretty high that you’ve all at some point used some form of a logger, be it log4j, its .Net port log4net, Serilog, Elmah or a number of other logging frameworks that are out there. At their core, one thing all these libraries have in common is their focus one one thing. Logging messages. Be it to a flat file, RDBMS, other or other forms of persistence. At no point do these libraries attempt to do anything more than just that. Now a lot of you have also used the same libraries to do something seemingly different, such as  sending out emails for example. The beauty of their design however is that at their core the implementations of these libraries themselves do things at a slightly higher level of abstraction. Namely, they take data, including the message level / severity, to be logged (from a source) and deliver it to a sink (which in most cases happens to be a flat file). While writing to flat files is usually the default implementation of the data sink provided by the frameworks, at their true level of abstraction, they are simply conduits for data, providing solid low level implementations for supporting different logging levels based on configuration. Even the basic things that we take for granted, such as file lock management, rolling log files etc, aren’t truly in the hands of the core framework. These auxiliary features are handled by specific implementations of the log sinks, from DMBS connectors, to file writers, and even SMTP appenders. This brings us to the second principle.

Open-Closed Principle : Software should be open for extension and closed for modification. This weird sounding principle basically attempts to convey that code should be designed such that allows its behavior to be modified without actually modifying the source code for the class itself

Now, historically, this principle has been interpreted in a few different ways, but they have all relied on inheritance to achieve the purported goal.

One interpretation of this principle suggests that a class is ‘open’ for extension if its structure enables us to add new properties or functions in addition to the existing ones provided by the structure. The ‘closed’ part of this interpretation applies when the module by itself is available for other modules for use through its publicly available interface.

The other interpretation of the open closed principle (based on a polymorphic view) refers to the use of abstractions and interfaces to provide and extend certain behaviors, while keeping other (often core) implementation details hidden, and closed to change.

The logger example from the first solid principle is a prime example of this approach, where the implementations of the data sinks (often referred to as appenders) are often supported purely through configuration without touching the core conduit between the source and the sink. The fie appender writing to flat files however, still maintains the Single Responsibility principle by assuming responsibility for only writing to flat files, while also taking care of features and issues specific to flat files such as managing file locks, and often providing out of the box support for rolling log files that prevent log files from becoming too large to manage and view using ordinary text editors.

Liskov’s Substitution Principle : Child class object can be substituted for a base class variable. This one is a slightly more theoretical concept than the others, as unlike the others, it tries more to enforce consistency of behavior between parent and child classes than the syntax of the classes themselves.

The gist of the matter is that if S is a sub type of base type T, then all instances of T in a program should be replaceable with instances of type S without altering the desirable properties of the program. Note the use of the term ‘desirable’, which has a more semantic implication than, say the easier to observe consistency of method signatures that is often associated with overriding / hiding of method implementations in class hierarchies. Without going into the technical details of the precise rules proposed under this rule, it is safe to say that if you see a method or property in a sub type behaving drastically differently from the implementation of its parent, chances are that this principle is being violated in some form.

Interface Segregation : Many interfaces (client specific) is better than the single monolithic interface.

Dependency Inversion : Just depend on abstractions and not implementations.

Learning SOLID principles have changed my programming life and the real one. We shall see each of these principles in more detail in the coming posts.

Language of the Developers
 

There are lots of people who tell me that they need to understand specific design patterns (factory, observer, chain of responsibility etc). And after we spend quite some time discussing the topic they brought up, I ask them why they wanted to understand the original topic. The answers were quite surprising to me. Most of them said, “Design patterns are the trend now”. Some of them said, “I want to write good code”. Some of the candidly accepted “I was asked about this in an interview”.

Well, Design patterns are the new trend in the software industry and are being asked in most of the senior level technical interviews. For me, these reasons don’t qualify to take steps to understand design patterns. But the answer “I want to write good code” was quite interesting. Since a person wants to write good code, he/she is finding ways to improve it by applying patterns. Of course, Design patterns are good to understand and it saves time to know about how to solve a requirement/problem. If we take a step back and see if we really need to understand Design patterns to write good software, the answer is a big NO. Good software’s have emerged even before GoF patterns were documented. So why do we need to understand Design patterns?

When we write the code required to solve the problems, when we write optimal code, when we write code which is readable, testable and maintainable, we would have already applied the correct strategies/principles to solve the problem. But we should realize that when we write code we create a document not only for the compiler but also for other developers to read. How can we explain our code to another developer in a short time? We need a language !! Christopher Alexander (father of design patterns) calls this as the Pattern Language. A language that professional designers can use to communicate designs. He defines what are patterns with respect to this language.

The elements of this language are entities called patterns. Each pattern describes a problem that occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice. — Christopher Alexander

In Software terms, Design patterns are the constructs of a language, professional software developers can use to communicate their design. Until a developer needs to communicate their design/code, they need not know Design patterns! But developers communicate mostly with code. When the code communicates, patterns in the code will help the reader to quickly understand it.