Thursday, April 13, 2017

Beyond Cargo Cult Agile

Many organizations trying to become “agile” never get beyond the imitation and ceremony stage. They go through the motions hoping that following the forms will give them the results they want. 

They don’t go beyond the forms to think about why certain practices are recommended, and how to apply them to this team, or how to improve on those standard practices. 

It is a rare organization that looks at the leadership behaviors and management systems that drive unhelpful team attitudes. 

And most large IT companies often have a “playbook” or an “Agile SDLC” document. Rather than being a map to help teams explore, corporate culture makes it a straitjacket that must be followed exactly.


For Example, the Stand Up meeting


One example is the Daily Stand Up, or Daily Scrum. Each team member answers the three questions: Oh sorry, wrong three questions. Those might actually be useful. 

Rather, teams are supposed to answer, what did you do yesterday? What are you going to do today? What impediments do you have? 

This often devolves into the “ice cream for breakfast” meeting. Everyone just thinks of the most recent thing they did (that they can remember) and talks about that.

Why?


Some of it is the Memento Effect

A major culprit is the insidious Timesheet Culture: Accounting has invaded development and wants to know “utilization” – people have to “estimate hours” for a task, then “track hours” and then “re-estimate hours” if there is still work left to do. 

The Timesheet Culture demands individual silos of work and prevents any kind of pairing or swarming

The standup becomes a "gotcha" to invent a rationalization for “why did you charge 8 hours to this project” (Accounting has not allowed the Scrum Master to make an effective transition from being a Project Manager). 

And then there is the dreaded “assigning work”, where team members have little or no choice in what to work on.

But the underlying reason is that the stand up as it is has no value, it just another “ceremony” the team has to endure. 

Is there anything more boring than a status report?


How to fix it


The first question I ask teams is, what is the purpose of your Information Radiators? 

Shouldn’t an Information Radiator radiate maybe, I don’t know, information? 

How about if that information reflected the changes in status due to the work you did? 
What if the radiator was structured so that “at a glance” it was obvious what is different today from yesterday? 
What if everyone updated the Information Radiator as the work progressed? 
And what if part of the team working agreement was to look at the Information Radiator before the stand up? 

In other words, what if before the meeting everyone already knew what the status of work was?

Then let’s change the stand up to make it meaningful to the team:
  1. What did you learn about the work yesterday? 
  2. What did you learn about how to do the work? 
  3. What did you learn about ways the team can improve?

(“Learn” includes raising questions about standards, processes, as well as the work)

Or better yet, use the time to consider the work as a team:
  1. What is the most important thing for the team to work on today? 
  2. What is the key improvement the team should work on today? 
  3. What does the team need to learn today?

Or perhaps:
  1. What process needs to be improved? 
  2. Where is quality at risk? 
  3. What experiments should the team conduct to explore improvements or mitigate quality risks?

You may recognize these ideas from Lean: Focus on the process and getting value to the customer as a team – eliminating waste and improving flow.

Saturday, September 12, 2015

Capacity, Tasks and Hours

Stop thinking of capacity in terms of hours. Stop estimating task duration in hours. Stop planning sprints to achieve “full personnel utilization” (again in hours). Such thinking is a holdover from the steam-engine era project management mindset, when people needed to be told what to do and when to do it. It is not how Agile teams work.

What is Capacity for Agile Teams?

“Capacity” for Agile teams is their velocity (in relatively-sized story points) for Scrum teams, or their throughput (from cycle time) for Kanban teams. Both are measures of accomplished work output. That is, they indicate how quickly the team can deliver value. More importantly, each is an emergent measurement.

Emergent Measures

An Emergent Measure is one that is derived from facts. It is not based on plans, wishes, “percentage completes”, mandatory overtime, assigned work, or pounding shoes on tables. It comes from historical data. It comes from an honest acknowledgement of reality as it is.

An emergent measure emerges from accumulated data. It has statistical variation based on history. A scrum master’s concern is first to provide space and time for the team to get variance under control, and then to improve.

Capacity is derived from Working Software

The agile principle of “working software” is seen every day during the team’s standup, when each team member describes the work they completed. This is most easily done in terms of the tasks they have done. No customer cares how many hours were spent on a task. They only care if something was done or not. Do not waste time updating “hours spent” and “hours remaining”. That is no better than waterfall’s “percentage complete”. The only thing that matters is what is actually complete.

The Yoda Rule
This leads to the "Yoda Rule". By planning and communicating in terms of tasks to do and tasks done, the team thinks at a level above “how long” something will take and instead focuses on why it might take that long – because of the tasks that need to be accomplished.

If a story is blocked – tasks are not completing in a day – “hours left” means nothing. It does not communicate anything to team members who might swarm to help. The first question they will ask is “what needs to be done”?

Use Tasks in Sprint Planning

During Sprint Planning, stories should be broken down into less-than-a-day-long tasks. New teams will have to learn what this means. Experienced teams know the cultural norm: A task is what your peers won’t laugh at when you report completing it at the next stand up. (A legitimate accomplishment to describe at standup is the discovery of additional tasks that must be completed).

This will give a task count. Throw away your hours-based burndown chart and use a task-based burndown instead. This has the added advantage of making conversations with the Product Owner more meaningful. Instead of saying “we have 100 hours left”, you say “we have to do X, we have to do Y, etc”, giving the Product Owner a chance to say whether they really intended you to do X or Y.

Use of Hours is a Cognitive Fallacy

Humans are linguistic creatures. Humans are semantic creatures. Humans do not communicate well, if at all, in numbers. Humans do not think in numbers. That is why we use stories to describe the work to do. Humans do ok with relative comparisons (hotter/colder, bigger/smaller). Humans do ok with relative time (longer/shorter, before/after).

Using the semantics of task description to communicate work leverages humanity’s 50,000 years of storytelling experience. The tasks of the stories in a sprint form a narrative for the sprint. This narrative should make sense to the team.

Planning in hours is “magical thinking”

If you are an executive, a manager or a scrum master who thinks you need to see hours you are engaging in magical thinking. You have an Illusion of Control – that if a task is estimated to take X hours it will actually be done in X hours. The reality is, until it is finished it is not done. That’s it. Simplistic, but easy to understand. Don’t waste the team’s time overthinking this.

This illusion is hard to give up. 

But it up you must give, as Yoda might say. 

Thursday, November 14, 2013

For Your Convenience (or, why your hatred of Microsoft burns with the fire of a thousand exploding stars)

You notice it first when you open any of the Microsoft Office 2013 products. A flat, white, featureless expanse greets you. A harbinger of experience to come.

As you attempt to use familiar products, you find things missing. Maybe my feature is no longer the default, and I have to explicitly enable it, you naively think. After a few hours of scouring support sites and hapless google searches you find a post from a support person who explains that your feature is no longer available at all. Not even as an option. 

Why, you ask? For your convenience, is the answer.

In Word, there used to be an excellent right-menu feature applicable to misspelled words: Add to Autocorrect. If there was a word you consistently fumble-fingered, you could right-click on it and, with Add to Autocorrect, never have to worry about misspelling it ever again.

But with Word 2013, this handy feature was removed. It is not available as an option. It is not available as an extension. It is not available as an "app" from the "Office App Store". It is an ex-feature.

What is Microsoft's reason for doing this? "It was too confusing to have so many choices available on the right-menu". Really?? "Too confusing" to be able to do what you want to do??

If you ask them why this feature was removed, they will respond:

We recommend you submit feedback about the requirement to our product team. Many features of current programs were designed and upgraded based on customers’ feedback.

Today I ran into a similar dead feature in Office 365 Outlook Web App (OWA). I always use OWA if I need to have an in-house client email address. It keeps things clean and separate. OWA gradually improved over the years, reaching a pinnacle in 2011 with an interface effectively indistinguishable from the installed Outlook client.

But Office 365 Outlook has changed that. Email message display no longer has a toolbar where you can click an arrow for "next message". No longer can you move a message to a subfolder.

And no longer can you add a specific date/time reminder to the email. Think of it. A feature of email that has been around since the dawn of time has been removed, "for your convenience".

You can set a reminder on an email to be reminded "today" or "tomorrow" or "next week" whatever that means. But if the email is pertinent to something that needs to happen 8am Monday? Nope. Can't do that anymore.

What is Microsoft's recommendation if you want to set a specific date/time reminder? Do they have a work-around, perhaps? Maybe an app from the "Office App Store"?

Jason Jiang of Microsoft Support helpfully suggests:

We recommend you submit feedback about the requirement to our product team. Many features of current programs were designed and upgraded based on customers’ feedback.

Thanks, Jason. Really appreciate that helpful response.

Monday, October 21, 2013

"Time Reborn" by Lee Smolin

Einstein removed the idea of Time from physics, rendering it an artifact of our conscious. Professor Smolin proposes a different approach to the definition of reality with Time as a real thing, and Space is an artifact.

Science depends on predictions and repeatable, testable, falsifiable results. When assembling a theory that encompasses the entire universe you have a problem with repeatability—a single universe limits your sample size. In addition, any “lab” experiment we do will only be on an “isolated system”, while the real world is an “open system”. So those experiments test only approximations.

The two predominant current theories in Physics are String Theory and Quantum Mechanics. Are they enough to explain the universe? That is, can they “scale up” to become Theories of Everything?

From String Theory it can be derived that for a finite number of universes (around 10500) the cosmological constant is positive. In our universe the cosmological constant is positive, so we could exist in any one of those universes with some random probability. This result is pretty much what we expect from String Theory: useless confirmation of what we already know, with no prediction and no chance of falsification. However, a few years ago it was also derived from String Theory that when you figure out how many universes would have a negative cosmological constant, the number is infinite. Simply by random chance a universe would have an infinitely greater chance of a negative constant. So either we are extremely lucky, or String Theory is wrong.

Quantum Mechanics is basically a method of conducting experiments—the choices the experimenter makes influences the reality of the system being studied. But at the cosmological scale, who is the observer?

We get three clues about nature from Quantum Physics: incompatible questions (i.e., simultaneously measure position and momentum), entanglement (pairs of quantum systems can share properties while each system remains individually indefinite – you can ask a question about the pair that has an answer but the related question to one of them does not), and nonlocality (as long as neither “half” of an entangled system interacts with another system, each can separate and move a great distance apart).

Both of these are “approximate theories” in the sense that they “truncate” how “accurate” they are. For example, Newtonian Mechanics is an approximate theory and as far as it went, an “effective” theory.

What must be true for a Cosmological Theory:
·         Any new theory must contain what we already know about nature
·         The new theory must be scientific (testable, etc)
·         The new theory should answer they “Why these laws” question. In particular, the values of fundamental constants
·         The new theory should answer the “Why these initial conditions” question. That is, why our universe has its properties rather than those of an alternate universe
·         The new theory should not have to assume symmetries nor conservation laws
·         Principle of explanatory closure – no chain of explanation points outside the universe
·         Principle of sufficient reason – there should be an answer to any reasonable question we ask – does the new theory increase the number of questions we can answer
·         Principle of reciprocal action – nothing in the universe acts on other things without itself being acted upon
·         Principle of No Isolated Systems – there’s no such thing as a system in nature that’s isolated from influence by the rest of the Universe
·         Identity of the indiscernibles – two things that have the same relationships with everything else in the universe must actually be the same thing – this implies there can be no symmetries
·         Physical variables should describe evolving relationships between dynamical entities. There should be no fixed-background structures (including fixed “laws of nature”)
·         We can create (from entanglement) novel properties in nature
·         Principle of Precedence – the future (that is, the way the universe is expected to behave) resembles the past (i.e., physics is case-law)
·         Conway/Kochen theory of quantum “free-will”– for entangled, then separated systems that are then tested by experimenters who can “pick” their measurements, the “response” of those systems is also “free”
·         Principle of maximal freedom – quantum mechanics describes a universe in which you can make probabilistic predictions, but those systems have the maximal freedom as would be expected with probabilistic predictions
·         Principle of before and after – if you make a prediction and then test it, implicitly time exists to distinguish between before the experiment/test and after

Time is real
The artifacts of Quantum Mechanics, entanglement and non-locality, indicate it is a truncation applicable to small, isolated systems. Restoring the reality of time makes a new formulation of Quantum Mechanics possible.

We cannot locate an object at a point unless we have some way to specify that place. If there were two identical objects then there must be two identical places in the universe. Similarly there can not be two events in spacetime that have exactly the same observable properties. Thus every moment of time, and every place at every moment, is uniquely distinguishable.

Different “kinds” of time:
·         Cosmological arrow of time – universe is expanding
·         Thermodynamic arrow of time – left to their own, things become more disordered
·         Biological arrow of time – living things are born, grow up, and die
·         Electromagnetic arrow of time – what we “see” is the world from “the past” (light is “old” when it gets to us)
·         Gravitational arrow of time – gravity waves move to their effects take time to propagate
·         Black-hole arrow of time – absence of black holes in the early universe
What if “local space” is not what you can walk to next door, but what is “on the other end of your phone”? This is the idea of Disordered Locality and the Non-local link, in which Space is an emergent phenomenon, as seen in non-locality and entanglement.
At the fundamental level of description, at which instead of “space” there is a network of interactions with everything potentially connected to everything else, then quantum theory and space emerge together.

Summary of differences:
Space is Real
Time is Real
Time is an illusion. Truth and reality are timeless
Time is the most real aspect of our perception of the word. Everything that is true and real is such in a moment that is one of a succession of moments
Space and geometry are real
Space is emergent and approximate
Laws of nature are timeless and inexplicable, apart from selection by the anthropic principle
Laws of nature evolve in time and may be explained by their history
The future is determined by the laws of physics acting on the initial conditions of the universe
The future is not totally predictable, hence partly open
The history of the universe is, in all its aspects, identical to some mathematical object
Many regularities in nature can be modeled by mathematical theories. But not every property of nature has a mirror in mathematics
The universe is spatially infinity. Probabilistic predictions are problematic, because they come down to taking the ratio of two infinite quantities
The universe is spatially finite. Probabilities are ordinary relative frequencies
The initial singularity is the beginning of time (when “time” is defined at all) and is inexplicable
The Big Bang is actually a “bounce” which is to be explained by the history of the universe before it
Our observable universe is one of an infinite collection of simultaneously existing of unobservable universes
Our universe is a stage in a succession of eras of the universe. Fossils, or remnants, of previous eras may be observed in cosmological data
Equilibrium is the natural state and inevitable fate of the universe
Only small subsystems of our universe come to uniform equilibria; gravitationally bound systems evolve to heterogeneous structured configurations
The observed complexity and order of the universe is a random accident due to a rare statistical fluctuation
The universe naturally self-organizes to increasing levels of complexity, driven by gravitation
Quantum mechanics is the final theory and right interpretation is that there are an infinity of actually existing alternative histories
Quantum mechanics is an approximation of an unknown cosmological theory





Tuesday, September 10, 2013

Mastermind by Konnikova

“Mastermind: How to think like Sherlock Holmes” by Maria Konnikova
The proposal of this book is to help you “operationalize” the latest results from Cognitive Science to improve your “thinking skills” as seen in Sherlock Holmes.
Topics described include:
I.                    What do you pay attention to?
a.       What are you collecting in your “brain attic”?
II.                  How good are you at paying attention? How do you improve?
a.       Be Selective
b.      Be Objective
c.       Be Inclusive
d.      Be Engaged
III.                Creativity and Imagination
a.       Overcome Imaginative Doubt
b.      Importance of Distance (separating yourself from the problem in order to think)
                                                               i.      Distancing through unrelated activity
                                                             ii.      Distancing through actual distance
         1.       Distancing through mental techniques (e.g., Meditation)
c.       Importance of Curiosity and Play
                                                               i.      Active and Passive knowledge
IV.                Deduction
a.       Problems with simple deduction
                                                               i.      Taleb’s Narrative Fallacy
                                                             ii.      Learn to tell the crucial from the incidental
b.      Continuing education
                                                               i.      Moving from Mindlessness to Mindfulness
         1.       Explain simply to someone else (e.g., novice)
                                                             ii.      Perils of Overconfidence
         1.       How to tell when you are overconfident
V.                  Self-Knowledge
a.       “Know yourself” and your environment
b.      Observe—carefully and thoughtfully
c.       Imagine—remembering to claim the space you may not think you need
d.      Deduce—only from what you have observed, and nothing more
e.      Learn—from your failures as from your successes
f.        Keep a diary
VI.                Expectations of yourself
a.       We are prisoners of our knowledge and motivation
b.      Beware of the reckless mind
c.       Take on the mindset of a Hunter
d.      Keep your attention ever-ready
e.      Be appropriate for your environment
f.        Adaptability
g.       Acknowledge your limitations
h.      Learn to quiet your mind

i.         Maintain constant vigilance

Monday, March 25, 2013

A Technique for Producing Ideas

From "A Technique for Producing Ideas" by James Webb Young
Two Principles

  1.  An Idea is a new combination
  2. The ability to make new combinations is heightened by an ability to see relationships

Step One: Gather Raw Material

Specific: those pertaining to the item you are looking to generate items about, in detail
General: a continuous ongoing process of browsing and accumulating knowledge of every subject you could not be interested in.
An idea will result from a combination of specific item (product, people) information with general knowledge about life and events. Having a usable store of general material means continually seeking out and noting what you do not already know.

Step Two: Find Relationships

Seek and compile direct and oblique combinations of the gathered materials.

Direct: those that “make sense”
Oblique: those that you would not have considered, or better yet, consider “funny”

Step Three: Let it Percolate

Key to this step is element three of Cleese’s Creativity Keys: Time. Take time to endure the process past the easy and facile answers through to the truly creative ideas that come later. This may even mean “leaving it alone” for a while and coming back to it later, letting it percolate in your subconscious. During the break, do something that feeds your creativity.

Step Four: Seize the ideas as they arrive

Ideas will appear at times and places that you do not expect, and often in forms you do not recognize. They must be seized and recorded.

Step Five: Test your ideas for anti-fragility

Let respected and trusted others examine and manipulate your ideas. See what works and what it would take to make it work. Ideas that are fragile will crumble—good ideas will be anti-fragile and grow under the pressure of examination. Others will expand and strengthen them.

Monday, June 25, 2012

Why Managers Hate Test-Driven Development (TDD)

Why do managers (and many developers) hate test-driven development? Many developers hate the extra work: creating unit tests, mocking frameworks etc. And besides, their code always runs correctly the first time, right? But why do managers refuse it?

Consider the chief aim of the manager: to show progress to their boss. And consider the fundamental truth of all software: it will have to be tested.

A "test-free" schedule can show three months of effort to reach the System Testing phase. That is, after three months the manager can report "We're done with the coding, we're just in testing now". The fact that it now takes six months to "test" (and rebuild) the code is ok, because all managers know know that developers produce crap that has to be tested for a long time.

On the other hand, the TDD manager takes five months to reach the System Testing phase. Thanks to TDD, System Testing only takes one month. Delivery in six months instead of nine.

But for two months the non-TDD manager has been able to claim progress, while the TDD manager gets the stink-eye from the Big Boss.