When to use Agile?
 

Agile – The agile means able to move quickly and rapidly.  Precisely the ability to both create and respond to change in order to fulfill the fast paced dynamic business environment refers to agility.

Agile software development refers to a set of protocols for building software under which requirements and solutions evolve through the collaborative effort of self-organizing cross-functional teams. It advocates adaptive planning, early delivery, evolutionary development and continuous improvement, and it also encourages rapid and flexible response to change.

The big question is when to use Agile. The world is moving towards Agile and one of the study shows that majority of companies would be very much Agile by 2020 and if not, it would be very difficult for them to survive. Although Agile adds value to process stream in most case while there are certain situations where it may not be apt.

While moving towards agile, one should evaluate following answers to below questions

  1. Requirement Definition – Changing or Constant
  2. Experience and Skills of a Team – Highly skilled or Newbies
  3. Change – Frequent changes or Less
  4. Resources – Dedicated or floating
  5. Timelines – Fixed or Flexible
  6. Documentation – Less or more
  7. Customer Involvement – Continuous or Intermittent
  8. Physical Location of resources – Co-located or Distributed

The mission critical applications or products probably still better be on waterfall or traditional model. Probably the 8 points mentioned above should be given a thought before taking a decision of choosing the right set of processes

Agile Manifesto and Core Principles
 

The agile manifesto says

Individuals and interactions over processes and tools

When I first time read this statement, I was little puzzled in terms of identifying the focus or intention behind this starting point. I had following question in my mind

  • Should we give little/no preference to tools or processes. Or if we do, to what extent.
  • Why is communication is so important? Why can’t we get it done utilizing tools so called ‘Agile Tools’?
  • What does it mean by individuals?

The whole world make use of processes and tools. The tools like JIRA has gotten lots of popularity.  At a time you feel that things can just go right even though there are distributed teams or people don’t interact much. The real truth is agile focus on people, communication and internal interaction.  In order to understand the real intention just replace the word “over” with “before” and it would make better sense. One cannot replace processes and tools completely while the communication gets more priority.  The collaboration, interaction and internal trust makes self organizing teams. The lack of self organizing teams result in ‘Handicapped Agile’.

Working software over comprehensive documentation

The comparison here is with waterfall or traditional development methodology where the customer requirements are documented start to end with huge sets of artifacts. I have personally seen team spending more time in documentation over building software. It is virtually impossible to document every piece of requirement in the beginning for most projects. Moreover the sign off from customer is just a formality as customer is dreaming of working piece over what is there in requirements. The elaboration of requirements itself is complicated and further there are so many unknowns.  The Agile believes in incremental development  and continuous delivery of working software to ensure the work is being done with respect to customer expectations.

It is good idea to have limited documentations while this statement every agile developer has taken it so seriously and developed a myth there should not be any documentation to be created at all.   Having product document, user manual or specifying common issues etc. would help project to run smoother.
Customer collaboration over contract negotiation

There is a common myth what I have seen in most of the projects I have gotten into that is “We don’t need a contract with customer and we would do it as we move forward with development”. This is one of the most misunderstood statement among developers and folks getting on to Agile. We obviously need contract and while the customer involvement gets more priority. Let me give you an example to understand this better.

Assume that you want to build a connector to connect to your IAM product to customer application to perform provisioning. The product roadmap and release planning is over post feasibility. As per release planning we have divided the work into number of sprints say 2 weeks 24 sprints. At this point as we constantly showing the working product to customer, the changes are welcome. The scope creep can be managed in this case is either by removing something from sprint of similar size which is not being worked up or adding that item in future sprints (not the work in progress). This might result in adding one or two sprints to overall 24 week sprint (Possibly we can reduce that while this kind of chances are fat)

Adding one or two sprints or probably more won’t disturb customer as at the end he would have a solid product and moreover the changes are proposed by customer which could be because market dynamics have changes, customer has missed the requirements or could be anything else.
Responding to change over following a plan

The successful product matters over complicated plans. Agile is extremely adaptable and typically go with release schedules while each release comprises of one or more sprints. The water fall model simply follow the plan and any change in requirement is very tough to include and further adds lots of cost to it. The flexibility of managing scope creep makes Agile superior. The focus is always respond to change. I have personally seen 6 weeks sprints have changed to 20+ weeks while customer was really happy as he has gotten what he wanted.

Here you have principles behind Agile Manifesto

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

There are number of reasons

  • Customer gets confidence by constantly getting the working software.
  • The early time to market adds lots of value.
  • The market dynamics helps to make changes to products.
  • More than 60%+ software projects are scrapped before they are complete. This percentage can be reduced and further if you product is not good enough you got to know well in advance and you can take a call whether you want to continue or stop it.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

The changes are always welcome. The customer gets what he wants and it would not hurt developer or company building the software.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

You know where you stand and the same is with respect to customer. This is probably the biggest benefit of agile.

Business people and developers must work together daily throughout the project.

The biggest reason for most of the projects failure is the team working in isolation with no clue whether they are on track. Working closely with customer constantly not only gives confidence to customer while at the same time benefits to team to be aligned well with customer with respect to their requirements.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Continuous attention to technical excellence and good design enhances agility.

Simplicity–the art of maximizing the amount of work not done–is essential.

The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

After years in Agility
 

It’s been few years since I started following agile methodology. When I look back it’s interesting and happy journey. Was thinking about the book (User stories applied) which introduced me to Agile world. Back then, I read the book like a story book. Now I re-read the book after two years after practicing Agile. It was a fantastic read! I thought of sharing few fundamental things for being in an agile team.

  1. Team > Individual : When you are part of an Agile team, the focus shifts from individuals to the team . This will be a big change for those who come from waterfall model. Although during the standup meetings people talk about their achievements for the day and the next actions, it’s not the individuals work alone that make sprints successful. Things like, taking up pending tasks, helping team members complete tasks, pair-programming, looking for the overall success of the business etc. are thing which make significant impact than an individual’s ability to write optimal code/test.
  2. Where is the Design : When I came from the waterfall model, I was surprised that there is no definite design phase. But soon I realize that design is happening in iteration over a period of time. While being in a sprint, Design happens when we do sprint planning and tasking, design happens when we do the pair programming, it happens when we do TDD. This way, there is more scope for the developers to think about code and design, than a single monolithic time slot to think through design.
  3. Agility is different from being in an agile team : Having an agile mindset is completely different from being part of an Agile team. Being part of Agile team gives us leverage to do iterative development, do the ceremonies of agile process etc. But when we have not imbibed the principles and the manifesto behind Agile, it’s not Agility.
  4. Communication : Focus and Communication are fundamental to any successful sprint team. When there is no direct communication, thoughts & opinions are not converted to actions. Be it daily standup meetings or technical discussions/pair-programming/or during sprint planning, connected and collaborative people can only create beautiful software.
  5. Practice, practice, practice : There is no testing phase, there is no business verification phase, nor there is any formal review process. How can we sustain quality ? how can we insure reliability ? The answer to these questions come from the software development practices like TDD, BDD, emergent design, CI etc. Agile provides a way to use these practices. Apart from the acceptance criteria for a story, Definition of Done(DOD) governs the acceptance criteria for all stories. When we have DOD covering these practices, it becomes a standard for ensuring quality. There techniques and practices in combination with Agile process can help to deliver continuous business value while ensuring good quality.

There are an indefinite list of advantages for doing iterative development over waterfall and vice-versa. Although Agile, helps to deliver software faster to clients, there can be projects where regular waterfall model might help. We need to be choosy in selecting our process for each and every project that we work on. Provide your inputs and thoughts on this topic in comments.

Projects do FAIL. It’s NOT only Code
 

It is NOT only code

We recently failed in our product. The 6 persons effort for nearly 2 years went into nothing. Besides we had all kind of testing, functional, unit, integrations, load test, stress test etc, to protect the application from any kind of questioning. When the management and product team took a decision about scraping the product, the scrum team was awestruck.  Soon all water cooler conversations were only about this. Team retrospected a lot about what has just happened and discussed everything under the sun that could have been better. Below are few pointers from that.

crash

Projects and products do fail

Software industry exists and continues to get better just because someone is continuing to write good/better code and solving problems. If we see the history of this industry, we just had assembly language, then we created abstractions after abstractions and OOPS, functional etc. As we progressed and got better, our ability to focus on the primary objective seems to be lost ie Problem solving. So when software “just don’t work, get scrapped“. This is what most developers conclude when project get scraped.

Stakeholders need to be happy

A projects need support and buy-in from all part of the organization. This includes marketing, management, product org, and of-course the customers. Each of them have a role to play in-order to successfully deliver the product the to customer. When one of them does not align to the vision and mission of the Product, it will ultimately fail.

stakeholders

When People change, alignment also changes.

Lot many times, when senior management changes, the whole mission of the company or division also changes drastically. This might be because they bring a new perspective, ideas and ways of doing things.

change

Competing products

Product acquisition and market reaction to a new products can cause products to be abandoned. Its often survival of the fittest. As a developer, Competition is healthy when it does not affect my product drastically. But unfortunately, businesses can not function like that. Ultimately, the product which the customer wants the most wins.

competition

Great products may not sell

Developers often think we need to build a great product in-order to capture the market and excite the customer. Great products also need great sales people to sell them.

sell

Timing

Its all about timing!. One of my friend told me an example of touch-based Windows mobile which was available very long before iPhones and Android dominated the mobile industry. Even though the product was good, there were lots of timing issues the market was not ready, Touch was not affordable etc.

timing

SO on and On

There are various reasons because of which leads to a failed software project. I have listed some of them here, But the list is infinite. And of-course Code is one of them.

Maintaining IEnumerable-Yield Data Pipeline is Hard
 

C# and .Net had IEnumerable and yield since its early versions. And I think its one of the most misunderstood features of C#. This is because of the pattern of deferred/lazy execution implementation.  Before we delve deep into this let us understand the iterator model.

IEnumerable and IEnumerator:

This is the foundation of the C# Iterator pattern and there are a number of detailed articles about this by Eric Lippert. The simple example is here.

There is a basket full of Oranges.

 
public class Basket
{
    private readonly IEnumerable Oranges;

    public Basket(IEnumerable oranges)
    {
        Oranges = oranges;
    }

    public IEnumerable GetOranges()
    {
        return Oranges;
    }
}

public class Orange
{
    public int NumberOfSlices { get; set; }

    public void Peal()
    {
            
    }

    public void Consume()
    {
            
    }
}

And if we have to consume oranges, we will have to peel each one of them. So we can use a simple iterator pattern on them.

var basket = new Basket(new [] { new Orange(), new Orange(), new Orange() });
foreach (var orange in basket.GetOranges())
{
    orange.Peal();
    orange.Consume();
}

But the problem is, we need not consume all the oranges from the basket at one go (I am not Hungry ;-).
So how can I get one orange from the basket at a time? or whenever I am Hungry?

Yield:

C# Yield keyword provides an answer to this problem. Slightly changing the Basket class using Yield keyword. (signature of the GetOranges Method not changed)

public class Basket
{
    private readonly Orange[] _oranges;

    public Basket(Orange[] oranges)
    {
        _oranges = oranges;
    }

    public IEnumerable GetOranges()
    {
        for (int i = 0; i < _oranges.Length; i++)
        {
            yield return _oranges[i];
        }
    }
}

Now the basket will provide one Orange at a time. ie when MoveNext() Method of the IEnumerator is called. In other words, we have paused execution of the “for” loop in GetOranges method till the time the next Orange is required.

Or in more technical words, until you Enumerate, the data will not be fetched and once Enumerated, there is no data. It all boils down to “when you Enumerate”.

The Maintenance Problem:

This is tricky for developers who are inheriting an existing code base. A developer might accidentally add an Enumeration before the place where it actually has to. The next time or When the original Enumeration happens, it can not find any new data and the iterator just completes without any looping. This can happen very silently without any exceptions. So care to be taken to see where we are yielding and where we are Enumerating.

“Yield return” is a great feature in C#, but use it with caution.  Let me know your thoughts.

Team Code Reading
 

One of the very old practices in the software industry is Code Reviews. The general process is, a senior engineer providing feedback on how a code is written by another engineer. There are many issues we have observed with this approach and we have adopted something called Team Code Reading.

Approach of Team Code Reading

  • Trust: We trust the team members that they have written good code and they are free to check-in their code without someone’s comments. Although this is validated by our functional tests and Unit tests.
  • Team Code reading: We meet as a team to read/go through to each of the lines of code to understand it and try to be critical about it, irrespective of who has written it. The person who wrote the code would normally explain the business and technological background behind it.
  • TODO’s /User Stories: Outcome of code reading is mostly a TODO on the code or a user story on the product backlog.
  • No Finger pointing: Being Critical is fundamental about the code reading session. There are no fingers pointed at the person who wrote it originally. The team takes collective responsibility for the code.

Benefits of code reading:

  • Knowledge Normalization: Since all the team members are part of the code reading session, the product and technology knowledge gets normalized/spread across all the team members. For example, A junior/fresh grad will get lots of technical best practices from the code reading session.
  • Team bonding: Since there is no finger pointing, the team owns the problem in the code, it helps team bond well along with the code.
  • New ideas: Code reading session becomes a platform for brainstorming for new ideas as everyone is in thinking on critically about the product and code. Some innovative ideas come out and have become great product features.

It isn’t all Rosy                                             

There is a cost associated with each of the approaches. Below are the few costs that you have to pay if you are doing Team code reading.

  • The complete team’s time is required to do the team code reading. This is the cost that team members need to spend to get the product/business knowledge.
  • Distraction: There are chances that team gets distracted by some idea/problem. Scrum master or one of team member need to re-align the team towards code, every time there is a distraction.

Concluding remarks

There are always many ways to improve product and code. But the process that is followed drives the effectiveness of the outcome. I believe Team Code reading can greatly help improve the effectiveness of the team.

 

SOLID Principles
 

Many of the old programmers have adopted SOLID principles and it is taught in lots of schools these days. Yet I find many people not clearly understanding SOLID principles. I think these principles are fundamental to creating good object oriented software and every developer should learn them. So I have series of posts for these principles

Single Responsibility : A piece of software always have a single responsibility!
What this basically translates into is that a piece of code should have one and only one functionality. The corollary to this would be that  a piece of code should have only a single reason to change. Some advantages of this would be to ensure minimal change to existing classes in case of a change in requirements, which avoids larger testing efforts for a larger number of classes changed, which cuts down on testing efforts and increase turnaround times, which leads to lower overall cost of ownership for the code, which saves $$.

Take the example of a logging class. As a software developer, chances are pretty high that you’ve all at some point used some form of a logger, be it log4j, its .Net port log4net, Serilog, Elmah or a number of other logging frameworks that are out there. At their core, one thing all these libraries have in common is their focus one one thing. Logging messages. Be it to a flat file, RDBMS, other or other forms of persistence. At no point do these libraries attempt to do anything more than just that. Now a lot of you have also used the same libraries to do something seemingly different, such as  sending out emails for example. The beauty of their design however is that at their core the implementations of these libraries themselves do things at a slightly higher level of abstraction. Namely, they take data, including the message level / severity, to be logged (from a source) and deliver it to a sink (which in most cases happens to be a flat file). While writing to flat files is usually the default implementation of the data sink provided by the frameworks, at their true level of abstraction, they are simply conduits for data, providing solid low level implementations for supporting different logging levels based on configuration. Even the basic things that we take for granted, such as file lock management, rolling log files etc, aren’t truly in the hands of the core framework. These auxiliary features are handled by specific implementations of the log sinks, from DMBS connectors, to file writers, and even SMTP appenders. This brings us to the second principle.

Open-Closed Principle : Software should be open for extension and closed for modification. This weird sounding principle basically attempts to convey that code should be designed such that allows its behavior to be modified without actually modifying the source code for the class itself

Now, historically, this principle has been interpreted in a few different ways, but they have all relied on inheritance to achieve the purported goal.

One interpretation of this principle suggests that a class is ‘open’ for extension if its structure enables us to add new properties or functions in addition to the existing ones provided by the structure. The ‘closed’ part of this interpretation applies when the module by itself is available for other modules for use through its publicly available interface.

The other interpretation of the open closed principle (based on a polymorphic view) refers to the use of abstractions and interfaces to provide and extend certain behaviors, while keeping other (often core) implementation details hidden, and closed to change.

The logger example from the first solid principle is a prime example of this approach, where the implementations of the data sinks (often referred to as appenders) are often supported purely through configuration without touching the core conduit between the source and the sink. The fie appender writing to flat files however, still maintains the Single Responsibility principle by assuming responsibility for only writing to flat files, while also taking care of features and issues specific to flat files such as managing file locks, and often providing out of the box support for rolling log files that prevent log files from becoming too large to manage and view using ordinary text editors.

Liskov’s Substitution Principle : Child class object can be substituted for a base class variable. This one is a slightly more theoretical concept than the others, as unlike the others, it tries more to enforce consistency of behavior between parent and child classes than the syntax of the classes themselves.

The gist of the matter is that if S is a sub type of base type T, then all instances of T in a program should be replaceable with instances of type S without altering the desirable properties of the program. Note the use of the term ‘desirable’, which has a more semantic implication than, say the easier to observe consistency of method signatures that is often associated with overriding / hiding of method implementations in class hierarchies. Without going into the technical details of the precise rules proposed under this rule, it is safe to say that if you see a method or property in a sub type behaving drastically differently from the implementation of its parent, chances are that this principle is being violated in some form.

Interface Segregation: Many interfaces (client specific) is better than the single monolithic interface.

Dependency Inversion : Just depend on abstractions and not implementations.

Learning SOLID principles have changed my programming life and the real one. We shall see each of these principles in more detail in the coming posts.

Language of the Developers
 

There are lots of people who tell me that they need to understand specific design patterns (factory, observer, chain of responsibility etc). And after we spend quite some time discussing the topic they brought up, I ask them why they wanted to understand the original topic. The answers were quite surprising to me. Most of them said, “Design patterns are the trend now”. Some of them said, “I want to write good code”. Some of the candidly accepted “I was asked about this in an interview”.

Well, Design patterns are the new trend in the software industry and are being asked in most of the senior level technical interviews. For me, these reasons don’t qualify to take steps to understand design patterns. But the answer “I want to write good code” was quite interesting. Since a person wants to write good code, he/she is finding ways to improve it by applying patterns. Of course, Design patterns are good to understand and it saves time to know about how to solve a requirement/problem. If we take a step back and see if we really need to understand Design patterns to write good software, the answer is a big NO. Good software’s have emerged even before GoF patterns were documented. So why do we need to understand Design patterns?

When we write the code required to solve the problems, when we write optimal code, when we write code which is readable, testable and maintainable, we would have already applied the correct strategies/principles to solve the problem. But we should realize that when we write code we create a document not only for the compiler but also for other developers to read. How can we explain our code to another developer in a short time? We need a language !! Christopher Alexander (father of design patterns) calls this as the Pattern Language. A language that professional designers can use to communicate designs. He defines what are patterns with respect to this language.

The elements of this language are entities called patterns. Each pattern describes a problem that occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice. — Christopher Alexander

In Software terms, Design patterns are the constructs of a language, professional software developers can use to communicate their design. Until a developer needs to communicate their design/code, they need not know Design patterns! But developers communicate mostly with code. When the code communicates, patterns in the code will help the reader to quickly understand it.