Wednesday, July 15, 2009

The Problem of Change

Imagine you are a first-rate aerodynamic engineer. The year is 1946, and you have been creating the products that won the war. However, today your boss walks in and says the days of propellers are over: you are now to work on an entirely new technology: rockets. How will your organization adopt the new technology? What project should you work on first? What should you expect? What if it doesn't work?

This is the situation organizations around the world face every day. Many of them have been successful using previous conventional technologies. But now they hear about some new technology and wonder what should they do.

Of utmost importance is executive vision. The effort involved in any significant technological change is immense. Effective management requires the ability to manage change. The key to managing change is the vision of where you want to be. Without high-level management leadership the change will not happen. Management must support the change. There must be an understanding of the risks and benefits of the new technology, but more importantly, an understanding of the difficulties of this kind of change.

To return to the aerodynamics to rockets metaphor, management must understand that despite the organization's previous success, with the new technology they are re-starting at square one. Management should not expect the first project using the new technology to be a trip to the moon. Small scale experiments are necessary first.

Furthermore, management must expect a few projects to "blow up on the pad". An effective engineering organization uses these "failures" to learn and improve. Without an acceptance of the risk of failure, no progress will be made.

This leads to the next major area of importance: the engineering organization. The effective engineering organization is at its core a learning organization. The organization has management support to examine new technology to determine if it is worthwhile or merely the latest flash in the pan.

The organization must be process and people focused with a culture of improvement both continuous and discontinuous. It understands technology transfer and has effective means of communication, training, and people development.

Many of the management techniques that produced success previously will be applicable to the new technology. How do you know which are applicable and which are not? And of those that are, how do they apply? Determining the answers is part of the process of technological transition. Invariably mistakes will be made. These mistakes must be treated as learning opportunities, not blame opportunities or the transition will not succeed.

Those organizations that have management support and already have in place an engineering culture are much more capable of change and the adoption of new technology than those without. While seemingly obvious, too many executives do not understand that the key is not technological at all, but the very way they themselves do their job.

Monday, July 6, 2009

I See You

Back in the old days the science fiction super genius Damon Knight wrote a story called "I See You". He's one of the giants of the field, and wrote some amazing classics (for example, "To Serve Man").I don't know if you can get his collections anymore, but you can get an ebook at the EBookMall, which has the summary:

A machine is invented that can view any person, place, or thing at any time in the past or present anywhere in the universe. The implications for politics, personal privacy, crime and punishment, space exploration, and historical research are mind-boggling and brilliantly explored in this short story. Hugo Award Nominee

While we don't quite have the "anywhere in the universe" part yet, we are nearly there as far as "the present" and even the recent past.

All that is needed is a user interface to an abstraction layer over all the traffic cams, surveillance cameras, nanny cams, cell phone videos, paparazzi, drone cams, flikr's, street-views, facebook's, you-tube's, etc on the internet. And your friendly search engine, of course. You could pick a place, say Trafalgar Square, and a time. The new tool would stitch together all the video and image sources to allow you to change viewpoints and pivot around the action in 3-D.

While this is fun, combine it with face recognition software to take the idea of "following" someone to a whole new level of creepy stalking. Now you are not just waiting for the next 140 characters but can actually watch them as they type. Set alerts to be notified when they emerge into view of any camera anywhere in the world. Back up the action and watch from a different angle. Use video interpolation software to fill in any missing parts.

Now go back and read the story. Just add a little more programming and we're there.

Friday, June 26, 2009

Requirements generation using Remember the Future

My current client needed to get requirements for a new system from people who do not think in terms of functional requirements, use cases, or even user stories. Their understanding of the future hypothetical system was too vague and undefined for those levels of detail.

Instead, I decided to run a session of Remember the Future (RtF). Not only does RtF encourage vision creation, it is also inexpensive and simple to run. The client understood that RtF generates material that then needs to be collated and translated into more "usable" requirements.

The session had good participation from marketing communication, business, and front-line customer interaction folk. The client was surprised by the amount of material that was generated, with about a third of it being directly translatable into requirements.

One of the reasons why Remember the Future is good for this sort of situation where specific detailed requirements are problematic is that by looking back from the successful "reality" of the future and telling us in the past what it is like up there, it leverages the human mind's ability to confabulate.

While confabulation is distressing from a politician explaining their fiascos and amusing in a four-year-old explaining how the dog got on the roof, it forces a pure "blue sky" vision into specifics and tangible details.

We can then derive requirements from the specifics and go back to the participants to check against the "reality" that they imagined.

The material that was not applicable to requirements is also useful because it allows us to set and communicate system boundaries.

Friday, June 5, 2009

Innovation Games in Action

I had the opportunity to work directly with Luke Hohmann, founder of Enthiosys and inventor of Innovation Games, helping to facilitate the sessions he was conducting. The venue was the User Conference of one of his customers, a software company from Austin. The games used were 2020-Vision, Speedboat, and Buy a Feature.

There was a lot of preparation behind the scenes prior to the event. The users were being asked to provide feedback on the future direction of the product, so time was spent putting together lists of features that could be developed along with descriptions of them, etc.

2020-Vision was used with executive-level customers to prioritize features. This is a more facilitator-focused activity, where participants compare features in pairs deciding "better One, better Two" much like your optometrist does when fitting you for new glasses.

Speedboat was used to identify things in the existing product that people wanted changed or improved. This game begins as an individual-focused activity, as each participant begins by writing down things that they personally don't like. As the game progresses and people see what other people have said more and more interaction occurs.

Buy a Feature is a completely participant-focused activity that simulates a "market" for features that "cost money". Users discuss the features and debate what is worth spending money on. This forces people to combine their money to get expensive features beyond the budget of any one person. This game is a cognitive decision-making triumph, as it leverages Wisdom of Crowds, Prediction Markets, and fun.

At the event was a journalist/observer who noted that despite the apparent chaos of the activities, it was actually quite directed and successful. My response was that Innovation Games events are forms of dynamic, non-linear stability, which is much more robust in the face of random or unexpected events. (The standard example is that of earthquake resistant buildings--fixed and rigid "stability" or designed to sway with the movement of the earth?)

Wednesday, May 27, 2009

A Quantum Entanglement test for Sentience

Quantum Entanglement was called "spooky action at a distance" by Einstein. For example you can "split" a photon and create two entangled photons, separate them by a great distance, and then what happens to one happens instantly to the other. "Instantly" means faster than it would take information traveling at the speed of light to get from one to the other.

What is interesting for us today is that the "happening" is done by a conscious mind. A photographic plate is examined, a phase filter is set, etc. In every article I've seen, the conscious mind is a human mind.

For purposes of this argument we'll call humans sentient. And we'll say the collapse/resolution of the quantum state is due to examination by a sentient being.

So here is my question. What if you trained an animal to "examine" the quantum state? Maybe with different lights for the state being up/down, and the rat or the bird or the dog has to push a lever to indicate they have "seen" the resolution one way or the other.

If the animal mind does cause the state to collapse, then do we say the animal is sentient? Or is it simply that being able to collapse a quantum state is not proof of sentience?

If the animal does not collapse the state then it is easier to say they are not sentient.

Wednesday, May 20, 2009

An Introduction to Time Travel using Black Holes for new Star Trek fans

People new to science fiction, black holes, or Star Trek might be confused by the new Star Trek movie. "How can we see Spock's mom in episodes of The Original Series (TOS) if she died on Vulcan?" Explanations that glibly proclaim "time travel" don't really explain what happened.

"Time Travel" using a black hole should not be called time travel at all, but instead alternate universe travel.

When you traverse a hyperbolic orbit around a black hole (and cross the event horizon) you "cease to exist" in the same universe you were in prior to the transit. Current physics says you end up in the "future", but what's the fun in that?

Think about what the word "horizon" means. As you watch something go "over the horizon" it has gone beyond your possible influence. Even if you do go to the future, it is a future that has no connection to the universe you left.

So in the movie we have Nero and Spock leaving their universe (the universe of William Shatner, TOS, The Next Generation (TNG), et al) and entering a new universe. This new universe is Our Universe. A universe with Nokia phones and 236 year old records of the Beastie Boys.

A universe where none of the episodes of The Original Series (TOS) exist.

Not merely don't happen, or haven't happened yet or will happen. Not exist.

This is not "time travel", because it has nothing in common with the previous, pre-transit universe. Nero and Spock have "stepped outside" the light cone (the set of all states reachable at the speed of light) of their universe and entered ours. (see The Large Scale Structure of Space-Time by Hawking et al page 169).

Any and all things you know of Star Trek occured in a different universe. There is no possible event path from this universe (with Bud Classic and dilithium crystal mines in Iowa) to a universe in which any event from the previous TV shows or movies occur.

You should think of TOS, TNG, DSN, V and the excreable Enterprise as just dreams, sort of like Dorothy and Oz.

Wednesday, May 13, 2009

Are you smarter than a cat?

Most people understand that we are prisoners of our own biases, of our perceptions. People who are supposed to be creative are often told to "think outside the box" (which is one of my most hated management cliches. It is always hard to "think outside the box" if upper management keeps nailing the lid shut on you). Our perceptions control us at a very fundamental level.

One of my neurocybernetics professors conducted experiments with kittens (yes, I know. He has since been sentenced for his crimes). One experiment involved raising the kittens in a black and white environment where there were no vertical lines. Only horizontal lines were present. When they were old enough they were brought into a normal environment.

Not surprisingly, they were not able to perceive vertical objects--they would wander around and bump into table legs or chairs. Such was their fate for being held inside the box of their stunted visual cortex.

But here is the real point. After a few weeks in a normal environment the neural plasticity of their brain saved them. They "learned" that vertical objects existed, and needed to be paid attention to. In other words, their biases were overcome by real world facts.

This is the real test of intelligence. Not whether you know everything, or if your "box" is the right box, but will you let facts change your learned behaviors? Or will your reflexes keep you walking into table legs?

Cats are smart enough to do this. Are you?

Monday, May 11, 2009

Creativity in Business

Innovation Games leverage the essential elements of creativity in business. There are five such elements* managers can provide their subordinates to enable creativity: Space, Time, Time, Confidence and Humor.

Space. An area away from phones, interruptions, and distractions must be provided. It helps to have plenty of writing or doodling room. Creative thinking can not be disturbed.

Time. Enough time must be set aside for the creative process to begin. An hour and a half is good. Once you have retreated to your safe creative space you will find you'll spend the first few minutes worrying about all the things you have to do. This is fine --- let these anxieties pass. Then you will be able to think about your problem clearly.

Time. You must give yourself (or your group) enough time to reach the best solution. This may mean multiple creative sessions (Cleese's example was that the shooting script for "A Fish Called Wanda" took 13 drafts). You will feel on edge because you haven't solved the problem. You will want to take the first semi-reasonable solution and go with it. You must resist this and maintain the tension of the unresolved problem until you reach the best solution, not just an adequate one.

Confidence/Permission. Criticism is not allowed in creative sessions. Allow all ideas to be put into play. Allow junior members to speak first, and don't allow senior member to ridicule them. People must feel they can take chances with incomplete or unreferenced thought. As ideas are tossed around, allow the occasional illogical transition. This is often the key step to a new view of the problem and a better solution.

Humor. Humor is an essential part of creativity. A main element of humor is the unexpected connection of two unrelated concepts. This is also a key element of creativity. Enabling humor starts the brain's analogizing mechanism. Some groups use computer generated random juxtapositions of key words just to stimulate thought.

So, the five elements of enabling creativity are: a space/time bubble in which to think, give yourself enough time to reach the best solution, empower the participants, and relax enough to play and have fun with the ideas.

*Taken from a talk given by John Cleese

Thursday, May 7, 2009

An Even More Obscure Tech Cult

I was recently developing a simulator for a board game to teach kids about the Futures Market. Since it was a board game, the simulator had to be realizable with dice, tokens, chips, etc. But it also needed to capture enough essential market behaviors to make it "realistic": trends, volatility (and changes in volatility), reversals, Fibonacci retracements, etc.

Whatever the physical mechanics and rules, I needed to be able to make long duration (thousands of turns) "runs" of the simulation and check its trajectory against the desired behaviors (and make sure it didn't always go to zero or infinity). How could I do this without actually playing the game?

What I needed was an interactive mathematical workspace. And an ability to quickly write little routines to "play" in that workspace. (and see if they could crash it)

What I needed was APL (APL Programming Language).

I first learned APL in High School. We had a 33 baud teletype connection to a CDC mainframe that ran APL (in addition to the Basic and FORTRAN that students were being subjected to). This version of APL used three-letter codes for the operators--$RO, $IO, $EP, etc. Despite that limitation, the clunky SpaceWar game I was trying to write in FORTRAN suddenly became trivial.

Later in college (1974) we had real IBM systems that ran APL/SV. I even purchased an APL typeball to use on the campus Seletric terminals. Besides its ability to rapid prototype, the "SV" in APL/SV meant "shared variables"--between users! Which of course pointed to the obvious practical application, real-time multiplayer combat simulation games.

It was only in graduate school that I finally used APL for something other than games. For my Masters thesis I used APL to develop adaptive algorithms (AKA neural nets) to do cryptanalysis (which earned my faculty advisor a visit from the Carter-era NSA when the work was published).

And where is APL now? It turns out to be still alive and well. That is, for an obscure language cult.

I downloaded a free version that uses the standard IBM keymap, dug out my copy of Pakin's APL\360 Reference Manual and quickly got my simulation up and debugged.

So after all these years I'm still using APL to play games.

Tuesday, May 5, 2009

Tech Cult Number Five

Infoworld has a list of the "top seven" tech cults. Number Five is "The Order of the Lisp".

"Like warrior monks driven into hiding, the Order of the Lisp was once a powerful force that lived at the heart of next-generation computing. Closely allied with artificial intelligence and expert systems, the Lisp (or List Processing) language fell into disrepute as those concepts became allied with the dark side in the late 1970s."

I was one of those who went over to "the dark side", working on a number of DARPA Strategic Computing Projects in the 1980's. In fact, one of my proudest moments was being on a "How the Military is Corrupting AI" panel at the American Association for Artificial Intelligence conference in Seattle, and being honored as the "Darth Vader of AI" by Norm Sondheimer of ISI.

Smarter people in our lab, such as Rick Saenz insisted that Lisp in and of itself was a useful thing. He tried to convince Management that we could/should sell a Lisp Development envirionment and build consumer products using Lisp.

I still tell people interested in learning to program that they should learn Lisp first, then all other languages are just a peculiar implementation of the concepts you will acquire.

Lisp teaches you the creation and manipulation of abstractions. That's what all programming is. Lisp gives you the purest way to express abstractions, and the easiest method to manipulate them.

Tuesday, April 21, 2009

Why "Games"?

Why are Innovation Games "games"?

There are two modes of operation in an organization, Open and Closed. The Open mode is essential for creativity. It invites thought, relaxed contemplation, and examination of ideas. The Closed mode is fine once you have solved your problem and have chosen a course of action: without the Closed mode nothing would get done. But an organization that stays in the Closed mode will fail to improve and will miss creative solutions to its problems. Using games captures two of the five essential elements of creativity: Confidence/Permission, and Humor.

Confidence/Permission: Criticism is not allowed in creative sessions. Allow all ideas to be put into play. Allow junior members to speak first, and don't allow senior member to ridicule them. People must feel they can take chances with incomplete or unreferenced thought. As ideas are tossed around, allow the ccasional illogical transition. This is often the key step to a new view of the problem and a better solution.

Humor: Humor is an essential part of creativity. A main element of humor is the unexpected connection of two unrelated concepts. This is also a key element of creativity. Enabling humor starts the brain's analogizing mechanism.

Often organizations try to eliminate humor because they feel it detracts from proper decorum, denigrates those in charge and leads to insubordination. This attitude comes from a confusion between "serious" and "solemn". It is possible to be serious about something and still maintain a sense of humor about it. And if you desire creative solutions, you MUST maintain a sense of humor. Organizations that insist on solemnity will forbid humor in all forms, keep in the Closed mode, and outlaw creativity. What you will find in these organizations is that people's natural humor and creativity, channeled away from the business process, will leak out in underground ways --- usually in jokes directed at the organization's management.

Friday, April 17, 2009

Innovation Games

Creativity is a learned skill. Group creativity requires training and coaching.

Recently I had a chance to go through certification in facilitating Innovation Games. Innovation Games are a set of 12 facilitated collaborative activities designed to help identify customer needs, define requirements, understand product/service usage, and design future products and services.

The different activities, or "games" address different aspects of the creative process: generating ideas, sorting ideas, prioritizing ideas, and pruning ideas.

As important as the activities themselves are, are the processes behind them: determining which activity (or set of activity) to conduct, how to prepare, how to monitor, how to organize, how to complete and how to extract maximum value.

The games can be used across strategic, tactical and operational horizons.

Why games? More on that later.

Wednesday, April 15, 2009

Iterative Incremental Processes (continued)

The Psychology of Iteration (part 2): Cybernetics and Control

John Cleese (yes that John Cleese) tells the story of his "favorite childhood superhero, Gordon the Guided Missle". In each episode, Gordon would be given a task. Setting out to accomplish this task, he would be constantly told how he was wrong: too low, too high, target to the left, target to the right, etc. He was always wrong. Always mistaken. However, he never despaired, and grew more confident with every correction. Finally, at the end of every episode he would reach his target successfully, and everyone was happy.

The lesson Cleese wants you to take is that it was because Gordon was willing to be "continually" wrong that he was ultimately successful. Gordon's "Process for Success" depended on feedback. It depended on "tracking the moving target" by constantly changing his direction. Viewed from the larger perspective, Gordon's ability to consistently deliver successful results was more stable because he included feedback and a willingness to admit mistakes. Even more so, an eagerness to make as many small mistakes as quickly as possible before they became big mistakes.

The theory behind system stability from feedback goes back to Von Neumann's Cybernetics and the original work on System Theory. Systems that can flex are inherently better than those that are rigid. You can see this in everything from ancient Japanese earthquake-resistant buildings to metabolic mechanisms to eco-systems to market economies.

Why should an endeavor as complicated as buiding software be any different?

You know you are going to get feedback. Do you unit test? Of course. Why? To feedback the test results into better code. Does your team do regular system builds? Pre-production tests? Beta releases? Customer-reviewed prototypes? Feedback, feedback, feedback, feedback.

Grab hold of this live wire. Consciously use this power.

But this can be scary. What keeps you from being electrocuted instead? What about the "feedback" when you put the microphone in front of the speaker? Tulip-mania and the dot-com bubble? Haven't you seen the Tacoma Narrows Bridge video (

Won't constant feedback, from customers or users, just as easily lead to cost overruns? Drive the project "into the ditch"? Everytime I talk to customers they add more features! And we have a fixed-price contract for a specific deliverable!

As with any tool, feedback can burn your hand or cook your dinner. Software development must use controlled feedback. It will depend on being explicit about the why's and what kinds of feedback you solicit.

As I hear the doubts ("so, how do I control this feedback?") I ask you to consider the following "homework" question: What keeps your unit testing from eventually generating an infinite number of bugs? Why doesn't each "fix" create even more bugs?

John Cleese' quotes from his speech on "The Importance of Mistakes"; any errors or misinterpretations are mine.
Padulo and Arbib's "System Theory" was my basic text on linear system theory.

Tuesday, April 14, 2009

The Rise of the Machines

Back in the days of refrigerator-sized Lisp Machines, one of the people in the AI Research lab was developing a text editor. A text editor that checked your spelling after each "space" and your grammar after each punctuation mark.

That's ridiculous, everyone said, because the whole machine would completely bog down when those checks were made. The project was scrapped.

Flash forward to computing power that makes such spell and grammar checks trivial. This is the sort of exponential change that people like Ray Kurzweil talk about.

Personally I don't think AI will get to the point of "real" intelligence (whatever that is), but will be able to quite convincingly fake it.

And, if instead of refrigerator-sized Lisp Machines we can house the software in machines that look like Tricia Helfer or Summer Glau, count me in.

Monday, April 13, 2009

Iterative Incremental Processes

There are still organizations that claim they do not use an iterative development process. Why is that? Wasn't this argument settled long ago?

The so-called non-iterative methods, such as waterfall, are most often used in organizations where the dominating culture is that of Mistake-Avoidance. This can come from a number of sources: don't waste the taxpayer's money, we can't tell the business people we made a mistake, or simply management that has never developed software using modern IDE's.

Non-iterative methods are used even in organizations that do not manage by fear. In talking to the IT department of a southeast asian central bank, I discovered they were happy and well adjusted, but insisted they did not use iteration. Projects lasted between 9 to 36 months. Delving deeper, it turned out that, yes, once fielded they did get feedback from users of changes, fixes, and enhancements, and yes, they collected those over a period of 6 or so months, at which point they would often start a new version of the application. My claim was that their iterations were just very, very, long.

Short iterations (e.g., daily) are empowered by modern IDE's. Iterative developement is essential for delivery of effective software. Management's avoidance of iteration centers around a fear of chaos and loss of control. "How do you know you are done?", "How do you budget and staff the project?", etc.

The analogy I like to use is that of the Sailboat and the Locomotive. Iterative development is captaining a sailboat---a method designed to handle changes in wind direction, currents, crew, revised destination, etc. Non-iterative development is a locomotive, meant to be predictable and repeatable. It is possible that in a restricted domain and problem space that software development could be organized to run like a train schedule. Beware if there is ever a bridge out, though.

Software professionals need to think, plan, act and manage more as sailors than conductors.