Agile Manifesto and Core Principles
 

The agile manifesto says

Individuals and interactions over processes and tools

When I first time read this statement, I was little puzzled in terms of identifying the focus or intention behind this starting point. I had following question in my mind

  • Should we give little/no preference to tools or processes. Or if we do, to what extent.
  • Why is communication is so important? Why can’t we get it done utilizing tools so called ‘Agile Tools’?
  • What does it mean by individuals?

The whole world make use of processes and tools. The tools like JIRA has gotten lots of popularity.  At a time you feel that things can just go right even though there are distributed teams or people don’t interact much. The real truth is agile focus on people, communication and internal interaction.  In order to understand the real intention just replace the word “over” with “before” and it would make better sense. One cannot replace processes and tools completely while the communication gets more priority.  The collaboration, interaction and internal trust makes self organizing teams. The lack of self organizing teams result in ‘Handicapped Agile’.

Working software over comprehensive documentation

The comparison here is with waterfall or traditional development methodology where the customer requirements are documented start to end with huge sets of artifacts. I have personally seen team spending more time in documentation over building software. It is virtually impossible to document every piece of requirement in the beginning for most projects. Moreover the sign off from customer is just a formality as customer is dreaming of working piece over what is there in requirements. The elaboration of requirements itself is complicated and further there are so many unknowns.  The Agile believes in incremental development  and continuous delivery of working software to ensure the work is being done with respect to customer expectations.

It is good idea to have limited documentations while this statement every agile developer has taken it so seriously and developed a myth there should not be any documentation to be created at all.   Having product document, user manual or specifying common issues etc. would help project to run smoother.
Customer collaboration over contract negotiation

There is a common myth what I have seen in most of the projects I have gotten into that is “We don’t need a contract with customer and we would do it as we move forward with development”. This is one of the most misunderstood statement among developers and folks getting on to Agile. We obviously need contract and while the customer involvement gets more priority. Let me give you an example to understand this better.

Assume that you want to build a connector to connect to your IAM product to customer application to perform provisioning. The product roadmap and release planning is over post feasibility. As per release planning we have divided the work into number of sprints say 2 weeks 24 sprints. At this point as we constantly showing the working product to customer, the changes are welcome. The scope creep can be managed in this case is either by removing something from sprint of similar size which is not being worked up or adding that item in future sprints (not the work in progress). This might result in adding one or two sprints to overall 24 week sprint (Possibly we can reduce that while this kind of chances are fat)

Adding one or two sprints or probably more won’t disturb customer as at the end he would have a solid product and moreover the changes are proposed by customer which could be because market dynamics have changes, customer has missed the requirements or could be anything else.
Responding to change over following a plan

The successful product matters over complicated plans. Agile is extremely adaptable and typically go with release schedules while each release comprises of one or more sprints. The water fall model simply follow the plan and any change in requirement is very tough to include and further adds lots of cost to it. The flexibility of managing scope creep makes Agile superior. The focus is always respond to change. I have personally seen 6 weeks sprints have changed to 20+ weeks while customer was really happy as he has gotten what he wanted.

Here you have principles behind Agile Manifesto

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

There are number of reasons

  • Customer gets confidence by constantly getting the working software.
  • The early time to market adds lots of value.
  • The market dynamics helps to make changes to products.
  • More than 60%+ software projects are scrapped before they are complete. This percentage can be reduced and further if you product is not good enough you got to know well in advance and you can take a call whether you want to continue or stop it.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

The changes are always welcome. The customer gets what he wants and it would not hurt developer or company building the software.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

You know where you stand and the same is with respect to customer. This is probably the biggest benefit of agile.

Business people and developers must work together daily throughout the project.

The biggest reason for most of the projects failure is the team working in isolation with no clue whether they are on track. Working closely with customer constantly not only gives confidence to customer while at the same time benefits to team to be aligned well with customer with respect to their requirements.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Continuous attention to technical excellence and good design enhances agility.

Simplicity–the art of maximizing the amount of work not done–is essential.

The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

After years in Agility
 

It’s been few years since I started following agile methodology. When I look back it’s interesting and happy journey. Was thinking about the book (User stories applied) which introduced me to Agile world. Back then, I read the book like a story book. Now I re-read the book after two years after practicing Agile. It was a fantastic read! I thought of sharing few fundamental things for being in an agile team.

  1. Team > Individual : When you are part of an Agile team, the focus shifts from individuals to the team . This will be a big change for those who come from waterfall model. Although during the standup meetings people talk about their achievements for the day and the next actions, it’s not the individuals work alone that make sprints successful. Things like, taking up pending tasks, helping team members complete tasks, pair-programming, looking for the overall success of the business etc. are thing which make significant impact than an individual’s ability to write optimal code/test.
  2. Where is the Design : When I came from the waterfall model, I was surprised that there is no definite design phase. But soon I realize that design is happening in iteration over a period of time. While being in a sprint, Design happens when we do sprint planning and tasking, design happens when we do the pair programming, it happens when we do TDD. This way, there is more scope for the developers to think about code and design, than a single monolithic time slot to think through design.
  3. Agility is different from being in an agile team : Having an agile mindset is completely different from being part of an Agile team. Being part of Agile team gives us leverage to do iterative development, do the ceremonies of agile process etc. But when we have not imbibed the principles and the manifesto behind Agile, it’s not Agility.
  4. Communication : Focus and Communication are fundamental to any successful sprint team. When there is no direct communication, thoughts & opinions are not converted to actions. Be it daily standup meetings or technical discussions/pair-programming/or during sprint planning, connected and collaborative people can only create beautiful software.
  5. Practice, practice, practice : There is no testing phase, there is no business verification phase, nor there is any formal review process. How can we sustain quality ? how can we insure reliability ? The answer to these questions come from the software development practices like TDD, BDD, emergent design, CI etc. Agile provides a way to use these practices. Apart from the acceptance criteria for a story, Definition of Done(DOD) governs the acceptance criteria for all stories. When we have DOD covering these practices, it becomes a standard for ensuring quality. There techniques and practices in combination with Agile process can help to deliver continuous business value while ensuring good quality.

There are an indefinite list of advantages for doing iterative development over waterfall and vice-versa. Although Agile, helps to deliver software faster to clients, there can be projects where regular waterfall model might help. We need to be choosy in selecting our process for each and every project that we work on. Provide your inputs and thoughts on this topic in comments.

What is “Pair Programming”? Is it worth doing it?
 

Pair programming is an agile software development technique in which two programmers work together at one workstation and share the same keyboard. One person (Driver) writes the code while the other person (Navigator) reviews it and at the same time thinks about the big picture.

How to do it right

  • Understand the requirement well before you start. Spend few minutes and discuss with each other.
  • Agree on one small goal at a time.
  • Support each other
    • If you are a driver, focus on small tasks and quickly complete it avoid bigger issues.  Trust navigator to your safety gate.
    • If you are a navigator, constantly review the code and think of a big picture. You don’t need to dictate the code.
  • Celebrate victory when the task is completed or you resolve a problem.
  • Changing roles few times in a day helps (Driver to Navigator and vice versa)

Do’s

  • Encourage pairing. Do not worry about slight productivity loss in beginning. This is something I have seen many teams when new teams are formed as they don’t have experience in the pairing.
  • Start with a trial. Do not force individuals to pair. Let them decide who wants to pair with whom.  Changing pairs constantly helps.
  • Pairing for more than six hours a day is not advisable.
  • Individuals should switch driver and navigator role when they get bored.
  • The Large monitors and good leg room are essential along with co-location which is absolutely mandatory.
  • Trust and support each other.  The team culture plays a critical role.

Benefits

  • Improves software quality without impacting time to deliver.
  • The focus and energy involved are much higher hence chances of making mistakes reduces significantly.
  • The better articulation of complexities and hidden details of coding tasks reduces the human errors.
  • The requirement elaboration and definition of done usually be better understood.
  • Build trust and help individuals to be better skilled.
  • The partners can switch when frustrated or stuck. The work doesn’t get compromised. The change in responsibilities once again adds energy to work.
  • New recruits come up to speed more rapidly in a pairing environment.
  • From a developer standpoint, pairing is enjoyable and valuable activity. I have seen developers who resisted pairing the first time, eventually loved it and found it to be much really useful in terms of learning, more engagement, better quality and successful careers.

Challenges

  • It’s social skill and it takes a time to do it. The best pair programmers exactly know when to say let’s try your idea first. Do not expect the outcome of pair programming from day one. It takes some time to do it in right way.
  • No benefits are expected if both the programmers are not actively engaging themselves. It’s “programming it loud’ methodology” hence it is essential that the driver and navigator constantly communicating. The silence kills the benefits of pairing.
  • If the two people have personal challenges, the pairing cannot be forced upon. The trust and mutual understanding between the two people is absolutely necessary
  • Experience mismatch is another bigger challenge. The senior programmers often want to have more control and give a little room to junior programmers.
  • View pairing as one person watching, the other person doing the actual work. That becomes boring and disengages the person watching, eliminating any real benefit from this practice.
  • Pairing should be avoided for very simple tasks or tasks which are very clear and can be done in little time.
  • Pairing needs to be done by two. The moment the third person added to it,  it cannot be called pairing anymore.
  • The co-location is absolutely mandatory. The absence of co-location where the work is shared between two programmers is called sharing over pairing.
  • Force the pairs or identify them ahead of time which may not be right in many scenarios. The best approach is to let the pairs form and swap by their own.

Myths

  • The pair programming is mentoring while it should never be. Programmers often pair with somebody with the goal to learn technology and domain. That is called either as mentoring or knowledge transfer. This should be treated as knowledge transition and expecting less than one person productivity is a fair expectation.
  • Two people working on the same story but different tasks individually is considered pairing while IT IS NOT! It is sharing the work but not pairing.
  • I have often seen individuals complete half the task and hand that over to another team member in evening sitting in a different country and assume that it’s pairing. This is again sharing of work and comes with extra cost due to unknowns, handoff and understanding each other’s work.
  • Doing agile requires pair programming. The reality is Agile manifesto never talks about pair programming.
  • The initial resistance of programmers that pairing is not a right thing to do. Most programmers like it when they try it.  Others don’t do it right and start believing it’s a waste.
  • The pairing would reduce the productivity to half. This is one of the most debated topics.  I have personally experienced that when it is done in right way, it improves the overall productivity of more than two people working individually.
What is an Agile Roadmap? What happens before sprint starts?
 

The agile roadmap is often confused with product roadmap or scrum ceremonies. The agile roadmap is indeed much beyond that. If it is not just the scrum activities then the question arises in terms of what is agile roadmap all about?  How important is it to know what happens before sprint planning? For most of us, sprint planning is the beginning of agile project while it is not (Practically it can be considered as mid-stage). A lot happen before ‘sprint’ ceremonies.   The agile roadmap would probably answer the entire process that is being followed to accomplish the project.

At a very abstract level, the agile roadmap has 7 phases listed as below:-

Vision

  • Product Owners in conjunction with senior leadership identifies the product vision.
  • The product vision talks about – What your product is, and what it entails? How it would support your company or organization strategy and who is going to consume it.

There are many more factors being considered here including the market, complexity, feasibility etc. Typically in this phase there is little or no involvement for the core development team.

Best practices

  • Touch base with your management and be updated with the vision of the product.
  • Provide inputs whenever applicable.

Product Roadmap

  • This phase defines high level of product requirements which are written at high level (EPICs).
  • Typically the EPICs are elaboration to have clarity while the stories may or may not be written.
  • The prioritization of EPICs happen as well in this phase.
  • The discussion between product owner and other stakeholders enable PO to define
    • High level estimates (Probably ROM -> +/- 30%).
    • Priorities

Best practices

  • Try to get involved in roadmap discussions.
  • Stay in touch with product management and ensure you know whats coming. What the organization goal is and what can we do to ensure we meet it.
  • Provide as much inputs as possible to product management to ensure the EPICs are well elaborated and understood.

Release Planning

  • The EPICs are further broken down into individual stories.
  • An agile project will have multiple releases with the highest priority features being picked up in order.
  • During this phase release timing for the specific product is determined.
  • The release plan must have to be created at the beginning of the project.
  • The number of sprints, team staffing and capacity is being looked at as well.
  • The estimation here is going to be +/- 10%. This is very important as the exact cost/schedule of the product/project is determined at the phase.

Best Practices

  • Every agile team must put in efforts to get involved in this phase.  The solid understanding in this phase is core to success of the product/project.
  • Identify impediments and provide inputs for estimation.

Sprint Planning/Grooming

  • The sprint team works with PO to groom the requirements based on priority.
  • Technical and product dependencies are discussed.
  • Individual tasks are created for each requirement. The next level of estimates are created (+/- 20%). Planning poker, relative estimate, and WBD are quite commonly used to provide an estimation. Based on the maturity or project and organization in the specific area, complexity based estimation can be utilized as well.
  • The team gives commitment by looking at the capacity and next level of estimates.

Please click here to get more details

Best practices

  • If the requirements are clearly stated or acceptance criteria is not written ask PO to do the same.
  • The grooming should be a constant activity and should be done for most of the stories prior to sprint start date.
  • The goal of the planning is to ensure everyone is aligned with details for each story, estimation and sprint commitment.  The planning can be done a day before sprint start date or on sprint start date.
  • The sprint commitment, assumptions and high level discussions should be sent to whole team.
  • The previous sprint must be closed before the next sprint start date. In addition to that the summary of the previous sprint to be sent to all the team members.
  • There should be a goal of N+2 sprint readiness which is to ensure that 2 sprint worth of work is groomed in advanced.
  • Make use of capacity planner for sprint commitment.

Daily Stand-ups

  • It should get completed in less than 10 minutes.
  • This meeting should answer
    • What did you do yesterday?
    • What are you going to do today?
    • Any impediment? If yes, then what is that?
  • Please note that standup meetings are not planning or technical discussion meetings. These are neither meant for micromanagement nor a meeting to provide a status.

Please click here to get more details

Best Practices

  • The standup should be time-bound. It should get completed in 10 minutes for less for 10 member team.
  • Any design or technical discussion should happen post standup. Not everyone needs to participate unless absolute necessary.
  • The standup should happen in morning. If there is a dependency with onshore counterpart, it should happen twice in a day.

Sprint Demo

  • This is essentially the demonstration of the working product (or “showcase”). This should happen post QA or peer testing.

Please click here to get more details

Best practices

  • This is a continuous activity and should be done as and when the story is done-done.
  • No story should be mark done without a demo. In case if the PO is not available, you should give the demo to rest of the team, record the session and share with a PO.

Retrospective

  • This happens at the end of the sprint. All the stakeholders sit together and discuss
    • Things which worked well
    • Areas for Improvement or things which we should stop doing it
    • Action Items
  • This meeting should also look at the action items of the previous retrospective and see where we stand. If the previous action items are not addressed, those should be included in current retrospective.

Please click here to get more details

Best practices

  • The retro must happen after every sprint. The retro should begin with action items of previous sprint.
  • There should be owner for each action item.
  • Be open. Appreciate people and discuss areas of improvement/things which did not work.

Many times the first three phases are ignored by agile teams.  The ultimate success requires or expects that the entire agile team to be well versed with first 3 phases as well. They should be part of it from day one.

Common best practices

  • Be open and honest. Support team work over individual contribution.
  • More the co-location better the outcome. Avoid working from home and silos.
  • Constantly review burn down chart to see where we stand.
  • Pick approx. 20% of tech debt in every sprint.
  • Focus on continuous improvement. That’s good for a company and for your growth.
  • Go to with pair programming wherever there is a possibility.
Choosing between Trunk based development(TBD) vs Feature branching
 

In a development environment like scrum, XP the branching strategy can significantly impact the overall speed of delivery of the product. There has been detailed documentation on the various branching strategies, and you can encounter TBD (Trunk based development), feature branching. In this article,  am covering the various factors that can impact the success or failure of a these branching strategy.

What is feature branching:

In this method, there is a code branch created for each and every feature. In agile based environments, typically, a branch is created for every story in the sprint.

 

What is Trunk based Development (TBD): When a single trunk/branch is used to create all the features of the product, then its called Trunk based development. Often a release branch is created when ever there is a release that is pushed to production.

Following are the various factors to look at before selecting Trunk based vs feature branching.

  • location of team members: when team members are co-located, the trunk is the best bet, helps us to get faster feedback. Also team can just talk about the code changes directly. Since the feedback will be faster in a single trunk, the team need to sync up more frequently. Co-location enables this sync trunk changes to happen frequently and faster. When there is a distributed environment (often in our teams), feature branching works best.

  • speed development environment: In a high speed dev environment, the trunk just shines as there are less and less process overhead, like branching, Pull request, review etc. Feature branching shines well when there is less speed in churning code.

  • size of code base: Insanely higher number of code lines can be better handled using feature branching. But for smaller code base, the cost of creating branches etc can be higher.

In our team meetings, we often talk about the right tools for the problem at hand. And using the right branching strategy is an important decision. We often question ourselves if the strategy is right for the given epic/sprint. If the answer is NO for few sprints, we know, we need to change something!

Please comment and let us know your opinion.

Projects do FAIL. It’s NOT only Code
 

It is NOT only code

We recently failed in our product. The 6 persons effort for nearly 2 years went into nothing. Besides we had all kind of testing, functional, unit, integrations, load test, stress test etc, to protect the application from any kind of questioning. When the management and product team took a decision about scraping the product, the scrum team was awestruck.  Soon all water cooler conversations were only about this. Team retrospected a lot about what has just happened and discussed everything under the sun that could have been better. Below are few pointers from that.

crash

Projects and products do fail

Software industry exists and continues to get better just because someone is continuing to write good/better code and solving problems. If we see the history of this industry, we just had assembly language, then we created abstractions after abstractions and OOPS, functional etc. As we progressed and got better, our ability to focus on the primary objective seems to be lost ie Problem solving. So when software “just don’t work, get scrapped“. This is what most developers conclude when project get scraped.

Stakeholders need to be happy

A projects need support and buy-in from all part of the organization. This includes marketing, management, product org, and of-course the customers. Each of them have a role to play in-order to successfully deliver the product the to customer. When one of them does not align to the vision and mission of the Product, it will ultimately fail.

stakeholders

When People change, alignment also changes.

Lot many times, when senior management changes, the whole mission of the company or division also changes drastically. This might be because they bring a new perspective, ideas and ways of doing things.

change

Competing products

Product acquisition and market reaction to a new products can cause products to be abandoned. Its often survival of the fittest. As a developer, Competition is healthy when it does not affect my product drastically. But unfortunately, businesses can not function like that. Ultimately, the product which the customer wants the most wins.

competition

Great products may not sell

Developers often think we need to build a great product in-order to capture the market and excite the customer. Great products also need great sales people to sell them.

sell

Timing

Its all about timing!. One of my friend told me an example of touch-based Windows mobile which was available very long before iPhones and Android dominated the mobile industry. Even though the product was good, there were lots of timing issues the market was not ready, Touch was not affordable etc.

timing

SO on and On

There are various reasons because of which leads to a failed software project. I have listed some of them here, But the list is infinite. And of-course Code is one of them.

Maintaining IEnumerable-Yield Data Pipeline is Hard
 

C# and .Net had IEnumerable and yield since its early versions. And I think its one of the most misunderstood features of C#. This is because of the pattern of deferred/lazy execution implementation.  Before we delve deep into this let us understand the iterator model.

IEnumerable and IEnumerator:

This is the foundation of the C# Iterator pattern and there are a number of detailed articles about this by Eric Lippert. The simple example is here.

There is a basket full of Oranges.

 
public class Basket
{
    private readonly IEnumerable Oranges;

    public Basket(IEnumerable oranges)
    {
        Oranges = oranges;
    }

    public IEnumerable GetOranges()
    {
        return Oranges;
    }
}

public class Orange
{
    public int NumberOfSlices { get; set; }

    public void Peal()
    {
            
    }

    public void Consume()
    {
            
    }
}

And if we have to consume oranges, we will have to peel each one of them. So we can use a simple iterator pattern on them.

var basket = new Basket(new [] { new Orange(), new Orange(), new Orange() });
foreach (var orange in basket.GetOranges())
{
    orange.Peal();
    orange.Consume();
}

But the problem is, we need not consume all the oranges from the basket at one go (I am not Hungry ;-).
So how can I get one orange from the basket at a time? or whenever I am Hungry?

Yield:

C# Yield keyword provides an answer to this problem. Slightly changing the Basket class using Yield keyword. (signature of the GetOranges Method not changed)

public class Basket
{
    private readonly Orange[] _oranges;

    public Basket(Orange[] oranges)
    {
        _oranges = oranges;
    }

    public IEnumerable GetOranges()
    {
        for (int i = 0; i < _oranges.Length; i++)
        {
            yield return _oranges[i];
        }
    }
}

Now the basket will provide one Orange at a time. ie when MoveNext() Method of the IEnumerator is called. In other words, we have paused execution of the “for” loop in GetOranges method till the time the next Orange is required.

Or in more technical words, until you Enumerate, the data will not be fetched and once Enumerated, there is no data. It all boils down to “when you Enumerate”.

The Maintenance Problem:

This is tricky for developers who are inheriting an existing code base. A developer might accidentally add an Enumeration before the place where it actually has to. The next time or When the original Enumeration happens, it can not find any new data and the iterator just completes without any looping. This can happen very silently without any exceptions. So care to be taken to see where we are yielding and where we are Enumerating.

“Yield return” is a great feature in C#, but use it with caution.  Let me know your thoughts.

Team Code Reading
 

One of the very old practices in the software industry is Code Reviews. The general process is, a senior engineer providing feedback on how a code is written by another engineer. There are many issues we have observed with this approach and we have adopted something called Team Code Reading.

Approach of Team Code Reading

  • Trust: We trust the team members that they have written good code and they are free to check-in their code without someone’s comments. Although this is validated by our functional tests and Unit tests.
  • Team Code reading: We meet as a team to read/go through to each of the lines of code to understand it and try to be critical about it, irrespective of who has written it. The person who wrote the code would normally explain the business and technological background behind it.
  • TODO’s /User Stories: Outcome of code reading is mostly a TODO on the code or a user story on the product backlog.
  • No Finger pointing: Being Critical is fundamental about the code reading session. There are no fingers pointed at the person who wrote it originally. The team takes collective responsibility for the code.

Benefits of code reading:

  • Knowledge Normalization: Since all the team members are part of the code reading session, the product and technology knowledge gets normalized/spread across all the team members. For example, A junior/fresh grad will get lots of technical best practices from the code reading session.
  • Team bonding: Since there is no finger pointing, the team owns the problem in the code, it helps team bond well along with the code.
  • New ideas: Code reading session becomes a platform for brainstorming for new ideas as everyone is in thinking on critically about the product and code. Some innovative ideas come out and have become great product features.

It isn’t all Rosy                                             

There is a cost associated with each of the approaches. Below are the few costs that you have to pay if you are doing Team code reading.

  • The complete team’s time is required to do the team code reading. This is the cost that team members need to spend to get the product/business knowledge.
  • Distraction: There are chances that team gets distracted by some idea/problem. Scrum master or one of team member need to re-align the team towards code, every time there is a distraction.

Concluding remarks

There are always many ways to improve product and code. But the process that is followed drives the effectiveness of the outcome. I believe Team Code reading can greatly help improve the effectiveness of the team.