“Dale, I have an estimating problem,” Paul said.
“My product manager, Sandra, wants to know when we’ll be done testing,
and I don‘t know how to estimate that.”

“What do you mean by ‘done’?” I asked.

Paul opened his mouth to say something, then closed it.
After a moment he said, “That’s a good question.
I’m not sure.”

Clients often ask me to help with ‘estimating problems.’
Sometimes a few quick questions reveal that
the problem is not an estimating problem, but a clarity problem.
It’s difficult to estimate well when you’re
unclear what you‘re estimating.

If you’re having difficulty estimating,
first define the variable you’re estimating.
Ask yourself, What variable, exactly, am I estimating?
Write your definition down as clearly as you can.
Paul wants to estimate the date when we will be done testing.

Next, test the definition for fuzziness.
One effective test is the ‘Mary had a little lamb’ Heuristic
from Gause and Weinberg’s book Exploring Requirements.
To apply this heuristic, read your definition several times,
each time emphasizing a different word.
What does each emphasis suggest about the clarity of the definition?

Let’s try this with Paul’s definition.
Emphasize the first word: The
date when we will be done testing.
This emphasis suggests that there is a single date when the testing will be
complete.
Does Paul’s product manager need
a single date?
Maybe not.
Perhaps Paul could structure the testing in
phases and estimate each phase separately.
That would reduce the complexity of each estimating task.

Try the technique yourself.
Emphasize each of the next few words in Paul’s definition and notice
what each emphasis suggests about the variable Paul is being asked
to estimate.

I’ll skip to the word done:
The date when we will be done testing.
What does it mean to be done testing?
When I asked Paul this question, he didn’t have an answer.
Other clients answer the question, but are
immediately unsatisfied by what comes out of their mouths.
“We’re done when we’ve found all the bugs,”
or, “We’re done when we’ve tested everything.”
As people utter these answers out loud, they realize that they will
never test everything, and will never know when they’ve found all the
bugs.
If these are our criteria for
being ‘done,’ we’ll never be done.
No wonder it’s hard to estimate.
So how can we solve this ‘estimating problem’?

Before we can answer that, we have to explore a related fuzziness,
the last word of the definition: The date when we will be
done testing.
What does testing mean?
There are lots of possible meanings.
It might mean looking for defects.
It might mean exercising the
system to assess its correctness.
It might mean discovering information about the quality of the system.
It might mean other things.

Let’s see if any of these meanings help us understand what done means.
When will we be done identifying the system’s
defects, or assessing its correctness, or discovering information about its
quality?
I don’t see a clear stopping point for any of those tasks.
We could continue each indefinitely.
The ‘Mary had a little lamb’ Heuristic has helped us to pinpoint the
ambiguity, but not to resolve it.

To resolve the ambiguity,
we can apply another clarifying technique that I call The Value Question:
“If we did that, what would it do for us?”
If we tested well, what would that do for us?
What value do we want testing to produce?

I see testing as an information service.
A primary purpose is to inform product stakeholders, to answer their questions
about the quality of the system so that they can make good decisions.
We inform developers so that they can trace
defects to their origins and fix them.
We inform project managers so that they can steer project
activities.
We inform product managers
so that they can decide whether to put the system into production.

If testing is about answering stakeholders’ questions about the quality of the
system, we will want to know what their key questions are, and what decisions
they will make based on the answers.

Paul met with Sandra to identify her key decisions and questions.
Her primary decision was whether to put the system into production.
She was confident, from earlier testing, that the system’s functionality
was okay, but less confident about its performance.
Her most important question was, “Will the system support 1000
concurrent users?”
And right now she
wants Paul to estimate when he will be able to answer that question.

Paul now has a more specific variable to estimate:
The date when we will determine whether the system will support 1000
concurrent users.
Though there is plenty of ambiguity here (as you will see if you apply the
‘Mary had a little lamb’ Heuristic), this is far more specific than
the date when we will be done testing.
Paul was able to estimate, with reasonable confidence,
when he would be able to determine Sandra’s primary question.

Sandra had several other key questions about the quality of the system.
“Does the new version start up at least as
fast as the previous version?
Does the daily billing report run in less than five minutes?
As data volume approaches one billion
records, will performance be fast enough with our current computers?”
Paul was able to translate each of these
questions into a variable that he could estimate well.

Notice that, as often happens, in clarifying Paul’s ‘estimating problem’
we have shifted the definition of the problem.
Even if Paul answers all of Sandra’s questions, he still might not be
done testing.
For example, if he discovers that the system will support only 300
concurrent users, he has answered Sandra’s question, but there will
be more testing to do after the developers increase the system’s capacity.

As it turns out, what Sandra wanted to know was not when Paul would be done
testing, but when he would be able to answer her questions about
the system’s performance.
We have redefined the problem so that it is not only clearer and easier
for Paul to solve, but also more valuable to Sandra.

The next time you have trouble estimating, test for clarity.
Use the ‘Mary had a little lamb’ Heuristic,
The Value Question, and any other techniques you know to clarify the variable
you are estimating.
Clarifying the problem can take you a long way toward solving it.

© 2006 Dale Emery

"I’m stuck and I need your help," Susan said. Susan was
Director of Human Resources at a multinational paper company, and was
leading the effort to change the company’s structure from top-down
management to self-directed work teams. The transition involved
thousands of people, and they had been working on the effort for a
year.

Susan said, "Most people are really excited about what we’re
doing, but then there are the resisters." As she said this, she
motioned with her hands as if pushing away. "The resisters don’t
want anything to do with teams. Mostly, they are company veterans who
have been here more than twenty years and are near retirement. Every
time they come to a meeting, I already know what they are going to
say." Again she gestured pushing away.

Susan’s repeated use of word "resisters," and her
repeated gesture of pushing-away, suggested that she was stuck in a
story. The story of resistance is common one. Change agents often
tell the story as a blockbuster drama, complete with good guys with
admirable goals, bad guys with unsavory goals, and escalating
conflict in which the bad guys thwart the good guys at every turn.
The struggle continues until one side vanquishes the other, which
resolves the conflict and ends the story.

In the story of resistance, the good guys are, of course, the
change agents who tell the story. They want only good things for
their organization, and they can accomplish the good things only by
influencing others to change. The bad guys are the "resisters."
Because the story is told from the change agents’ point of view, we
don’t always know what the bad guys want, but we know it’s less
important than what the change agents want. And the conflict is the
resistance–the bad guys refuse to change, preventing the good guys’
from achieving their noble goals. As for the ending, we don’t know
how the story ends, because the story of resistance is usually told
in progress, in the middle of the struggle. But the storyteller, the
change agent, yearns for an ending in which the good guy triumphs by
overcoming the resistance.

Susan’s story was a mild one. She did not make the common mistake
of judging the "resisters" to be stupid, insincere,
incompetent, or mean-spirited. But her story hypnotized her into
seeing the resisters, and possibly herself, as something less than
whole people, as mere roles locked in opposition. The story
constrained her resourcefulness, compounded her frustration, and kept
her stuck.

When you’re stuck in a story of resistance, you can unblock
yourself by peeling away the constricting elements of the story. One
useful approach is to tell the story from a new point of view. How
would a neutral third party tell the story of change at Susan’s
company? What story would the company veterans tell?

Susan’s "resisters" might tell the story this way. We’ve
been working effectively for years. Now suddenly a bunch of nameless
bureaucrats from the central office arbitrarily decide that we have
to change everything and work in a whole new way. And they never even
asked our opinion.

A neutral third party might tell a different story. Susan led
a six-month task force, whose members included representatives from
every department, to determine how to decentralize decisions and give
more autonomy to the people who know the most about making the
product. The task force organized two pilot projects to test the idea
of self-organized teams. The pilot projects were highly successful,
and the task force planned to convert the entire manufacturing
process to self-organized teams. Some of the senior employees liked
the idea of increased autonomy, but were concerned about how such a
dramatic change would affect productivity. Though they expressed
their ambivalence loudly and repeatedly, Susan and the task force
ignored the veterans’ concerns, and moved ahead with the transition.

The point of this approach is not to predict accurately how others
would actually tell the story, but to recognize the range of stories
that different people might tell about the same events. If you can
see that many different stories are possible, and even reasonable,
you’re less likely to remain blocked by your old, constricting story.
That’s the time to ask the "resisters" to tell their story.
Then listen to the story they tell.

A related approach is to shift from heavy words to lighter words
to describe people’s roles and behaviors. "Resisters" is a
heavy, loaded word that suggests a limited range of behaviors and
intentions. What do resisters do? They resist. What are the
resisters’ intentions? To resist. Heavy words lock the story of
resistance in place.

When Susan finished telling me about the "resisters," I
invited her to use lighter words to tell a more resourceful story.
"Instead of calling these folks ‘resisters,’" I said,
"suppose you think of them as people who are resisting this
change at this time."

She considered this in silence for minute, then looked at me and
said, "Wow. That makes a big difference. When I think of them as
resisters, it’s as if I have them all figured out, that they’re just
resistant to change. When I think of them as resisting a particular
change at a particular time, I see them more as people. Maybe they
have reasons for resisting."

I said, "Now, instead of thinking of them as resisting
the change, what if you think of them as responding to it?"

Again she went silent. After a moment, she said, "Thank you!
Now I know what I need to do!"

I talked with Susan several months later. She had met several
times with the company veterans who were most concerned about the
change, and focused on listening carefully to what they had to say.
She learned that their biggest concern was how they would fit in. As
long-time employees, they had worked for years to earn the respect of
their colleagues. Their seniority was reinforced by the old top-down
structure. What would happen to their status, reputation, and
contribution as the company shifted to an unfamiliar way of working?

Susan’s old story of "resisters" fell away. After a few
long conversations, together she and the company veterans worked out
a solution. These senior employees would become mentors who would
help new employees learn "how we work in teams around here."
This solution met Susan’s need to move forward with the transition
and the company veterans’ need to play a significant, distinguished
role in the new organization.

Susan and the company veterans had co-authored a new story in
which everyone was a good guy. Such a story makes for a lousy
blockbuster movie. There’s too little drama. But Susan didn’t care
about drama. She cared about creating a more effective organization.