Bloated Domain Objects And CQRS (Command, Query Responsibility Segregation)
 

Problem of Bloated Domain objects

In business software applications, the domain objects (entities) are used to represent the business domain. As the application grows and adds more business logic, the service layer, mappers and other patterns gets applied. Often this leads to domain object becomes bloated and the related components become huge & un-maintainable.

CQRS solves the common problem of having a bloated Domain objects. The domain objects get bloated largely because of bounded context. The series of contexts which makes developers think that a single domain object is sufficient to handle all the related things. For example, a large Invoice object for handling Invoice, Shipment and handling change of address for customer . But in reality, these contexts (invoicing, shipment and change) need not be related to same Invoice entity.

What is Command, Query Responsibility segregation (CQRS)?

In order to simplify the Domain objects, CQRS proposes to have two types of domain entities.

  • those serving the command (ordering/assertion services)  – For example, SaveCustomer, CreateInvoice, ShipProduct etc
  • those serving a Query (request) – examples include GetCustomerAddress, SearchCustomer etc

With this separation, the complexity (number of fields, methods) of entities used becomes simplified. And hence the Data mapper layers & the service layers becomes more simplified.

Where can I use CQRS?

  • Largely complex system: Applying CQRS on a simple CRUD operation based system is a over kill. When there is a domain heavy system, like banking and financing systems, LOB applications where business logic, lots of boundary conditions are heavy. Where it makes DDD (Domain driven design) provides high value.
  • Situations where you will apply Microservices, Eventual consistency and Event Sourcing. When we have separation of concerns using CQRS, the microservices becomes much simpler to design and maintain. With Event sourcing we are focused on getting the data (query) from other related sources and is what CQRS propagates.

Final words

CQRS is a carefully thought out pattern for simplifying large and complex systems. And it should not be applied for simple green field applications. Do let me know what you think.

50 Web Performance Tips and Tricks
 

I came across the below video on the web performance improvement tips and tricks. This is fantastic.

http://channel9.msdn.com/Events/Build/2012/3-132

In this video, Jatinder talks about Six fundamental principles for improving web application performance. He also talks a lot about how we go about Decreasing CPU time and increasing parallelism on the client machine to achieve faster web performance.

While I went through the video, I captured all the tricks he talks about. And thought will be useful for others while they watch it. Please find below the tricks.

Quickly respond to network request

  • Avoid 3XX Redirections (63% of top websites use redirect)
  • Avoid Meta refresh
  • Minimize Server time for Requests
  • Use Content distribution Networks (CDN)
  • Maximize concurrent connections.
  • Reuse connections – don’t sent connection close.
  • Know your other servers – you are only fast as your weakest link
  • Understand your network timing

Minimize bytes downloaded

  • GZIP Compression Network traffic
  • Persist application resources locally (Windows 8 applications)
  • Cache dynamic resources in the application Cache (html5 app cache)
  • Provide cacheable content
  • Send Conditional request
  • Cache Data requests (jQuery AJAX calls)
  • Standardize File name capitalization convention.

Efficiently structure markup

  • Load pages in latest browser mode
  • Use http header to specify legacy IE mode
  • Link CSS in the top of the page header, never on the bottom.
  • Avoid using @import of hierarchical styles
  • avoid embedded and inline CSS
  • Only include necessary styles in the page.
  • Always link JS at the end of the page.
  • Avoid linking JS in the header (use the defer & async attribute)
  • Avoid Inline JS
  • Remove Duplicate code (52% of the web have duplicate code
  • Standardize on a single framework

Optimize your media usage

  • Avoid death by too many images
  • If possible use Image Sprites
  • Use png image file format
  • Use Native image resolutions
  • Replace Images with CSS3 Gradients, border Radius
  • Leverage CSS3 Transforms
  • Use Data URI’s for Small Single view images
  • Avoid complex SVG paths
  • Video : Use preview images
  • Minimize media plugin usage
  • Proactively download Future media

Write Fast JavaScript

  • Stick to Integer math (Math.Floor)
  • Minify your JavaScript
  • Initialize JavaScript on Demand
  • Minimize your DOM Interactions
  • Built in DOM methods always more efficient (firstchild, nextsibling methods are faster)
  • Use Selectors for Collection access (document.querySelectorAll)
  • Use .innerHTML to construct your page
  • Batch your markup changes
  • Maintain smaller DOM (less than 1000 elements)
  • Json always faster than XML
  • Use Native Json methods
  • Use regular expressions Sparingly

Know what your app is doing

  • Understand JavaScript Timers (setTimeout, SetInterval)
  • Combine Application Timers
  • Ensure dormant timers are not running
  • Align timers to display Frame (16.7)
  • Use window.requestAminationFrame(renderLoop) for Animations
  • Know when your application is visible (document.hidden, Visibilitychange (event))

Conclusion

The web optimization is not easy and needs exhaustive deep look and hopefully this check list helps while optimizing your pages. Enjoy coding high performing applications. If you have more tips please provide them in the comments.

 

Micro-Services, Eventual Consistency and Event sourcing patterns
 

Microservices is a really becoming a famous architectural pattern that most of the new software written these days, just apply them. One of the most important differentiation between the traditional web services and micro services pattern is the amount of common stuff across different subject areas. This calls for a discussion on how Eventual Consistency pattern is mandatory for successfully implementing micro services.

Micro-Service Pattern

Generally in a micro-service pattern, the API’s are split in to small subject areas. For example for a CRM application, the subject areas are

  • Customer information – like name, address, email, phone
  • Appointment information – which customer, sales person, when, where
  • Relationship management – sales/manager, what products, interests
  • Campaign data – offers, deals etc

Then micro-services are built for each of the subject areas.  The microservices are logically and physically separated from each other. ie there is no sharing (code, database, component etc) between any of these micro services of these subject areas. Pictorially its looks something like this.

Applying Eventual Consistency Pattern

In Micro-services, there is no data that is shared across the micro services. In order to synchronize the data across these isolated storages of these services, we need to apply the eventual consistency pattern. You can read more about applying the pattern correctly here. The simpler way we can achieve consistency across these micro-services is through Event Sourcing pattern.

Event Sourcing

Event sourcing is a process of capturing application state changes in the form of events.  An example of events are customer created, customer updated, Deal created, etc.  Other systems listen to these events and they take relevant action. You can read more about event sourcing here.

Conclusion

Event sourcing is the new way of storing changes to systems and help in making micro-services eventually consistent. These patterns together form well maintainable, reliable and scalable systems in the modern world.

BDD (Behavior Driven Development)
 

In the previous article we looked at how to do Test Driven Development (TDD) while doing the XP (Extreme Programming). In XP, the implementation of the tasks are done in the steering phase of the iteration planning. One of the tenets of the steering phase is “functional tests are run”. In this article we will see how to create functional tests using Behavior driven development (BDD) and its benefits.

Following are the some of the benefits of automated functional tests.

  • Functionality experienced by the User is tested
  • Saves testing time for the developers
  • Test in every environment (Dev, QA, Stage) without much effort
  • Generates confidence in every stake holder

Tools required for doing BDD in .Net

There are various tools available to do BDD in .Net. I have listed few of them below

SpecFlow is one of the most popular tools used in the .Net/C# community. And in this blog lets use the same. The tool comes as a Visual Studio Extension and can be downloaded directly from VS.

Creating a SpecFlow Project

Once SpecFlow extension is installed, the template required for doing BDD will be installed.

  • Create a simple “Class Library” Project in Visual studio
  • In the project, create a SpecFlow Feature file

Selecting the test framework

Before we Create/execute the scenario, we need to wire the SpecFlow with a compatible test framework. To do that we need to make changes to the app.Config file of the project.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />
  </configSections>
  <specFlow>
    <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
  <!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
    <unitTestProvider name="xUnit" />
  </specFlow>
</configuration>

In this example we are using XUnit as the testing framework. SpecFlow supports a series of testing frameworks and more details can be found here.

Creating the Behavior

Now, let us look at how to create a functional test using behavior driven approach. Let’s consider the following story while creating a POS (point of sale) application for a super market.

“As a Supermarket POS app developer,  i would like the API to Give the total amount while billing so that I can wire up API against the UI.

Let’s write the Behavior required in order to implement the above API requirement .

Feature: SuperMarketProductsAPI
	As a Supermarket Biller 
	i would like the API to
	Give the total amount while billing
@SmokeTest
Scenario: Find the total amount while billing
	Given Shopping cart is filled with all the items required by consumer
	And Campaign information for the month is available in the system
	When I pass the cart to create bill API
	Then the total bill amount should be calculated

The above specification is written using the custom language (testing DSL) create by the specflow.org. (Please read about what is DSL here.)

There are 2 parts to the above Specflow specification

  • Feature – Denotes bigger context (description/story) of the application
  • Scenario – Specific workflow or behavior of the system under the feature. Each scenario has the following sub-parts
    • Given, And – describes what we already assumed available
    • When – The specific action which will trigger the workflow/behavior
    • Then  – expected behavior

 

Creating step definitions

When we run the unit test corresponding to the above test we it will fail as there are no definitions corresponding to the above scenarios.

Now Right Click on the feature file and select “Generate step Definitions”

You will see the pop-up like below, select “Generate” and then “Save” the file.

The file will have the details about “what SpecFlow should do when test is executed”.

using System;
using TechTalk.SpecFlow;

namespace SuperMarketFunctionalTests
{
    [Binding]
    public class SuperMarketProductsAPISteps
    {
        [Given(@"Shopping cart is filled with all the items required by consumer")]
        public void GivenShoppingCartIsFilledWithAllTheItemsRequiredByConsumer()
        {
            ScenarioContext.Current.Pending();
        }
        
        [Given(@"Campaign information for the month is available in the system")]
        public void GivenCampaignInformationForTheMonthIsAvailableInTheSystem()
        {
            ScenarioContext.Current.Pending();
        }
        
        [When(@"I pass the cart to create bill API")]
        public void WhenIPassTheCartToCreateBillAPI()
        {
            ScenarioContext.Current.Pending();
        }
        
        [Then(@"the total bill amount should be calculated")]
        public void ThenTheTotalBillAmountShouldBeCalculated()
        {
            ScenarioContext.Current.Pending();
        }
    }
}

When you build the project and execute the test corresponding to this, it will fail. This is because none of the parts of the test (given, when, then) are having implementation.

Writing code to make it work

Now that the behavior has been created, we are good to validate that with the team and implement the code/functionality corresponding to the requirement. Of course using best coding practices like TDD 🙂

Making the behavior test pass

In order to make the behavior test pass we need to write the implementation in the “Step Definition” file.

namespace SuperMarketFunctionalTests
{
    using Newtonsoft.Json;
    using System.Collections.Generic;
    using System.Net.Http;
    using TechTalk.SpecFlow;
    using Xunit;

    [Binding]
    public class SuperMarketProductsAPISteps
    {
        Product P1 = new Product { Name = "test1", Cost = 5 };
        Product P2 = new Product { Name = "test2", Cost = 10 };
        ShoppingCart cart;
        List<Campaign> campaign;
        Bill bill;
        [Given(@"Shopping cart is filled with all the items required by consumer")]
        public void GivenShoppingCartIsFilledWithAllTheItemsRequiredByConsumer()
        {
            cart = new ShoppingCart { Products = new List<Product> { P1, P2 } };
        }
        
        [Given(@"Campaign information for the month is available in the system")]
        public void GivenCampaignInformationForTheMonthIsAvailableInTheSystem()
        {
            campaign = new List<Campaign> { new Campaign { product = P1, discount = 1 } };
        }
        
        [When(@"I pass the cart to create bill API")]
        public void WhenIPassTheCartToCreateBillAPI()
        {
            var client = new HttpClient();
            var response = client.PostAsync("http://myapi.supermarket.com", new StringContent(JsonConvert.SerializeObject(cart))).Result;
            var output = response.Content.ReadAsStringAsync().Result;
            bill = JsonConvert.DeserializeObject<Bill>(output);
        }
        
        [Then(@"the total bill amount should be calculated")]
        public void ThenTheTotalBillAmountShouldBeCalculated()
        {
            Assert.True(bill.totalAmount == 14);
        }
    }
}

Once the code is there we can see the behavior test pass.

Conclusion

BDD is very powerful way of ensuring high code quality in conjunction with business requirement. In Extreme programming (XP) , BDD is considered to be a very important practice where functional tests are mandated to run at least once during the iteration.  Is BDD/Functional tests helping your team/business in achieving success or not ? Do let us know your thoughts.

TDD (Test Driven development)
 

There are lots of practices that need to be followed while doing XP as an agile methodology. And one of them is automated testing. However, there is lots of confusion in the tech industry if you want to test the behavior of the system or the subunits/parts of the system. First, lets us understand these terminologies. In this post, we will talk about Test driven development.

TDD (Test Driven development)

TDD or Test Driven development is an approach in which unit tests drive the development of the code. That is, when there is a new feature that needs to be developed, the unit tests corresponding to the feature is developed first before the actual code need to create the feature is coded.

Let’s take an example. Let’s say at a point of sale system, we need to calculate the bill based on the list of products picked by the consumer. Let’s write unit test corresponding to this code. (I have used C# based XUnit as the unit testing framework)

        [Fact(DisplayName = "SimpleBiller Should Calculate Total bill Amount")]
        public void SimpleBillShouldCalculate()
        {
            //Given
            var ProductsList = new List();
            ProductsList.Add(new Product { Name = "test Product1", Price = 5 });
            ProductsList.Add(new Product { Name = "test Product2", Price = 5 });
            var simpleBill = new SimpleBiller();

            //When
            var bill = simpleBill.GenerateBill(ProductsList);

            //then
            Assert.Equal(10, bill.TotalCost);
        }

In the above Unit test, we have 3 parts.

  • Given
  • When
  • Then

Given

This is the known part of the problem. i.e in mathematical terms, it’s part of the problem. In the example, we have initialized the variables and known things corresponding to the products (selected by the consumer) and the Biller object.

When

This is the business action based on which we are writing the code. In this case its GenerateBill action/method.

Then

The purpose of the tests is to ensure that the code/action does it properly as per the plan. In order to do that, we are asserting the assumptions/result of the method we are testing. In our case, we are asserting the total amount that the bill will have to be generated for.

Running the test

There are three stages of running the test.

  • Red – When we run the test now, it will fail as there is no code corresponding to the calculate bill functionality.
  • Green – In order to fix the above test, let’s write the real code corresponding to the above unit test/requirement.
        public Bill GenerateBill(IEnumerable products)
        {
            var bill = new Bill { Products = products };
            foreach (var product in bill.Products)
            {
                bill.TotalCost += product.Price;
            }

            return bill;
        }

Now, when we call GenerateBill method from the test, it will return the Bill with the total amount.

  • Refactor – With the above code written corresponding to the test, we are sure the functionality is correct as per the requirement. But the code is not optimal, as the foreach loop can be reduced to a simple C# LINQ  expression.
        public Bill GenerateBill(IEnumerable products)
        {
            var bill = new Bill { Products = products };
            bill.TotalCost = bill.Products.Sum(x => x.Price);
            return bill;
        }

Conclusion

TDD is a foolproof approach to developing software as per the requirement. The test also provides confidence to the developers on the edge conditions and other possibilities in the code. However, there is an alternative thought that TDD could waste developer time. Do you think, TDD is the right approach to software development? Please provide your comments.

Choosing between Trunk based development(TBD) vs Feature branching
 

In a development environment like scrum, XP the branching strategy can significantly impact the overall speed of delivery of the product. There has been detailed documentation on the various branching strategies, and you can encounter TBD (Trunk based development), feature branching. In this article,  am covering the various factors that can impact the success or failure of a these branching strategy.

What is feature branching:

In this method, there is a code branch created for each and every feature. In agile based environments, typically, a branch is created for every story in the sprint.

 

What is Trunk based Development (TBD): When a single trunk/branch is used to create all the features of the product, then its called Trunk based development. Often a release branch is created when ever there is a release that is pushed to production.

Following are the various factors to look at before selecting Trunk based vs feature branching.

  • location of team members: when team members are co-located, the trunk is the best bet, helps us to get faster feedback. Also team can just talk about the code changes directly. Since the feedback will be faster in a single trunk, the team need to sync up more frequently. Co-location enables this sync trunk changes to happen frequently and faster. When there is a distributed environment (often in our teams), feature branching works best.

  • speed development environment: In a high speed dev environment, the trunk just shines as there are less and less process overhead, like branching, Pull request, review etc. Feature branching shines well when there is less speed in churning code.

  • size of code base: Insanely higher number of code lines can be better handled using feature branching. But for smaller code base, the cost of creating branches etc can be higher.

In our team meetings, we often talk about the right tools for the problem at hand. And using the right branching strategy is an important decision. We often question ourselves if the strategy is right for the given epic/sprint. If the answer is NO for few sprints, we know, we need to change something!

Please comment and let us know your opinion.

Projects do FAIL. It’s NOT only Code
 

It is NOT only code

We recently failed in our product. The 6 persons effort for nearly 2 years went into nothing. Besides we had all kind of testing, functional, unit, integrations, load test, stress test etc, to protect the application from any kind of questioning. When the management and product team took a decision about scraping the product, the scrum team was awestruck.  Soon all water cooler conversations were only about this. Team retrospected a lot about what has just happened and discussed everything under the sun that could have been better. Below are few pointers from that.

crash

Projects and products do fail

Software industry exists and continues to get better just because someone is continuing to write good/better code and solving problems. If we see the history of this industry, we just had assembly language, then we created abstractions after abstractions and OOPS, functional etc. As we progressed and got better, our ability to focus on the primary objective seems to be lost ie Problem solving. So when software “just don’t work, get scrapped“. This is what most developers conclude when project get scraped.

Stakeholders need to be happy

A projects need support and buy-in from all part of the organization. This includes marketing, management, product org, and of-course the customers. Each of them have a role to play in-order to successfully deliver the product the to customer. When one of them does not align to the vision and mission of the Product, it will ultimately fail.

stakeholders

When People change, alignment also changes.

Lot many times, when senior management changes, the whole mission of the company or division also changes drastically. This might be because they bring a new perspective, ideas and ways of doing things.

change

Competing products

Product acquisition and market reaction to a new products can cause products to be abandoned. Its often survival of the fittest. As a developer, Competition is healthy when it does not affect my product drastically. But unfortunately, businesses can not function like that. Ultimately, the product which the customer wants the most wins.

competition

Great products may not sell

Developers often think we need to build a great product in-order to capture the market and excite the customer. Great products also need great sales people to sell them.

sell

Timing

Its all about timing!. One of my friend told me an example of touch-based Windows mobile which was available very long before iPhones and Android dominated the mobile industry. Even though the product was good, there were lots of timing issues the market was not ready, Touch was not affordable etc.

timing

SO on and On

There are various reasons because of which leads to a failed software project. I have listed some of them here, But the list is infinite. And of-course Code is one of them.

Maintaining IEnumerable-Yield Data Pipeline is Hard
 

C# and .Net had IEnumerable and yield since its early versions. And I think its one of the most misunderstood features of C#. This is because of the pattern of deferred/lazy execution implementation.  Before we delve deep into this let us understand the iterator model.

IEnumerable and IEnumerator:

This is the foundation of the C# Iterator pattern and there are a number of detailed articles about this by Eric Lippert. The simple example is here.

There is a basket full of Oranges.

 
public class Basket
{
    private readonly IEnumerable Oranges;

    public Basket(IEnumerable oranges)
    {
        Oranges = oranges;
    }

    public IEnumerable GetOranges()
    {
        return Oranges;
    }
}

public class Orange
{
    public int NumberOfSlices { get; set; }

    public void Peal()
    {
            
    }

    public void Consume()
    {
            
    }
}

And if we have to consume oranges, we will have to peel each one of them. So we can use a simple iterator pattern on them.

var basket = new Basket(new [] { new Orange(), new Orange(), new Orange() });
foreach (var orange in basket.GetOranges())
{
    orange.Peal();
    orange.Consume();
}

But the problem is, we need not consume all the oranges from the basket at one go (I am not Hungry ;-).
So how can I get one orange from the basket at a time? or whenever I am Hungry?

Yield:

C# Yield keyword provides an answer to this problem. Slightly changing the Basket class using Yield keyword. (signature of the GetOranges Method not changed)

public class Basket
{
    private readonly Orange[] _oranges;

    public Basket(Orange[] oranges)
    {
        _oranges = oranges;
    }

    public IEnumerable GetOranges()
    {
        for (int i = 0; i < _oranges.Length; i++)
        {
            yield return _oranges[i];
        }
    }
}

Now the basket will provide one Orange at a time. ie when MoveNext() Method of the IEnumerator is called. In other words, we have paused execution of the “for” loop in GetOranges method till the time the next Orange is required.

Or in more technical words, until you Enumerate, the data will not be fetched and once Enumerated, there is no data. It all boils down to “when you Enumerate”.

The Maintenance Problem:

This is tricky for developers who are inheriting an existing code base. A developer might accidentally add an Enumeration before the place where it actually has to. The next time or When the original Enumeration happens, it can not find any new data and the iterator just completes without any looping. This can happen very silently without any exceptions. So care to be taken to see where we are yielding and where we are Enumerating.

“Yield return” is a great feature in C#, but use it with caution.  Let me know your thoughts.