No, this post has nothing to do with the files lost earlier in the week.  Rather, I’m writing to discuss the work of CAE, creators of very sophisticated simulation and modelling technologies software.  With annual revenues exceeding $1 billion, CAE employs approximately 6,000 people.

This large company appears to be very good at making good software.

CAE 5000 Series wins GOOD DESIGN award in the Transportation category, awarded by the Chicago Athenaeum (Museum of Architecture and Design) and the European Centre for Architecture Art Design and Urban Studies. GOOD DESIGN bestows international recognition upon designers and manufacturers for advancing new and innovative product concepts and for stretching the envelope beyond what is considered standard product and consumer design.

The 5000 Series is graphic modeling software ! When was the last time you heard of any kind of software winning an award for “stretching the envelope” for “product and consumer design”?

I saw some of their work at the recent Intelligent Systems Collaborative conference.  There they talked about a system they developed for cities’ emergency services dispatchers.  The concept of the system is to model the roads, traffic speeds, the locations of the first responders, and compute expected arrival times for any individual 911 call.

Aside: Modelling city traffic is in itself an ambitious project but by no means is this new.  Traffic departments across the country have been using such technology to help them decide where to put traffic lights, how to time them, and test road designs to see how well they will work for the city’s traffic problems.  In my city of Windsor, Ontario, this has been a major discussion point ever since US border security tightened up after 9/11, causing miles of truck backups on city streets.

CAE takes this general modeling information, and applies sophisticated algorithms to compute how well the city is covered by its first responders.  Quoting from their white paper:

An indication on the GUI alerts the Communications Officer that a redeployment of resources is required, and, within seconds of the user’s request for a recommendation, an optimized redeployment of available resources is presented to the user both textually and on the system’s map display.

The deployment optimization takes into consideration the real-time positioning and status of all available units through the tools integration with the automatic vehicle locator (AVL) and CAD systems.  In addition, a sophisticated traffic and routing analysis algorithm is used to generate the drive time for each unit deployment taking into consideration the time of day and historical traffic patterns.

In short, although human dispatchers are still responsible for all decisions concerning deployment, the CAE Deploy system works very hard to help the dispatcher understand what is the best way to deploy vehicles.  Often, this will mean instructing an ambulance to leave the garage and go to a mobile waiting position.  This could be to cover for another vehicle that has already been called into action, or to adjust for the constraints of rush-hour traffic.

As for the interface, CAE wasn’t haphazard about designing it.  Rather, they employed human-factors professionals to go through iterative evaluations of the GUI, working with the communications officers and paramedics during each phase of development.  The GUI they designed provides immediate access to the information these people need, ensures the officer is constantly aware of the current situation through the visual presentation, and provides all the necessary tools to operate effectively.

In deploying the deployment system, they were able to achieve an 80% savings in decision time by the officer, while at the same time dramatically reducing the cognitive workload demand from consistently overloaded, to  consistently managable.

And that’s the value of a collaborative system.

(I would add a link to the CAE Deploy product, but it seems they haven’t published that yet.  Instead, I’ll direct you to another interesting system.  This time, a system for disaster response simulation and management.)

Yesterday, as I working to set up WordPress on my site… I suddenly discovered that all my files got reverted to a state about three days old!

I didn’t realize there was any change management in effect.   In fact, I think there isn’t.  Anyhow, this meant WordPress got reverted to it’s freshly installed state, forgetting all the posts I had imported, the posts I had just written, and all my settings. My home page got changed back to the 4/28 version.  At least my uploaded WordPress templates didn’t get erased.

Now what could possibly have caused such an event without me knowing it? Here’s the suspects:

• Wordpress - perhaps there’s a reset button I triggered?
• CPanel - did somehow the file manager decide to back out of a few changes?
• SmartFTP - maybe it decided to sync my computer’s files to the server without me intending that?
• HostGator - obviously they could have reverted files to a previous day, although I can’t imagine why

None of these seem likely, so I’m stumped. What I can say is that a well designed collaborative system wouldn’t have done this.

In a collaborative system, if my action led to the erasure of 3 days work, the system would be aware of this and make sure I was aware also. I don’t mean something as stupid as a dialog box asking “Are you sure you want to do this?” on it. There are much more informative methods. The system could show explicitly the files to replaced. It could provide an easy way to backup the old files in case I didn’t like the results. UNDO would be an option.

As far as I can tell, none of the above programs even think to do this kind of stuff.  In fact, they weren’t designed to think.  They aren’t aware of the impact operations they are performing. Rather, they were programmed to just do a set of tasks, and not worry about anything else.  Like Users.

I spent most of last week at the Canadian AI/GI/CRV/IS/SmartLinkages conference (aka the 2008 Intelligent Systems Collaborative) , which just happened to be in my hometown this year. I met dozens of researchers who presented on a huge variety of intelligent systems (such as a dynamic ambulance dispatcher), and fundamental algorithms (eg. counting people in a video frame). I’ll report on several of these in the next week.

Dr. Sue Abu-Hakima, CEO of Amika Mobile, talked about the system they developed to make your ultra-thin SMS phone just a little smarter. What they’ve done is create a mail server that takes your plain ordinary mail (you know, that account that gets you 300 messages per day), and forwards it to your mobile phone as a text message.

Now, that would be rather awful if they sent every message, and the entire message every time. You would be receiving way too many messages. The messages would be way too long for you to grasp on the tiny 140 character screens. To solve this, Amika built some advanced filtering and compression algorithms. Their server decides which parts of the email are needed and only sends those parts. email address and subject are included. Quoted text, html formatting, and signatures are excluded. Then before sending, it abbreviates common words and expressions so it all fits.

Besides filtering message content, Amika also is selective in what messages it forwards. The server uses content analysis to decide whether the message is something you really need to see right away. If it’s not urgent, you won’t be bothered. You won’t even see the message until you get back to the office, and log into your computer. On the other hand, if the message is urgent, you can text message back to the server, and it will convert your message to an email back to the original sender.

There are of course all kinds of services that can be provided via SMS. I suppose what makes this one interesting is the way it actively helps you see just the info you need. It takes highly sophisticated algorithms to recognize “urgent” mail, and the result is a slick service that keeps you just a little more connected.

I’ve been extremely busy of late, planning a business, learning about more effective management, organizing myself, and attending a conference on emerging software technologies. I’ve spent a lot of time thinking about what exactly I wanted to communicate on this site, and how I should go about it. I’m now ready to get back into action.

Very shortly, this blog will be integrated with the rest of the collaborative-systems website. The result will be one site that simultaneously fills two roles: The first role will be a blog, continually updated with the latest ideas, links, and comments on the emerging world of collaborative software. The second role is an information resource where all the blog posts will be organized into a coherent structure, browsable from the home page. From there you will be able to learn all about collaborative systems, or learn in detail whatever aspect you choose to investigate.

As for content, you will soon find

• abundant examples of current systems
• explorations of what’s coming
• explanations of how these systems can be built.

In the near future, expect to see talk of

• collaborative chess playing,
• haptic (force-feedback) systems
• graphic interfaces
• artificial intelligence
• the future of wireless communication
• and more.

Stay tuned!

I added about a half dozen pages to the site today. Check it out if you’re interested. For my next update, I think next I’ll try to get more detailed, and include specific examples.

Computers are great tools which haven’t been fully exploited yet. So far software generally provides tools to enable you to tell the computer what you want, and the computer gives it to you. Word or Excel etc are gen purp apps that give you tools. SimplyAccounting and SolidWorks are more vertical market tools that permit more specialized operations, but they really don’t know what you are up to. QuickTax is a step better. Because tax forms are so standardized and the rules apply to everyone, software companies are able to create software which will figure out your taxes for you, and quickly. Because it knows the rules of tax declarations.

Knowledge is the future of software. If you could explain to a computer about the kind of work you do, tell it what your expectations are for the results, then you have a starting point. From that description we can start creating programs to help you get your work done. A computer program can “look over your shoulder” and tell you when you make a mistake. MS Word does this when you type something that isn’t a wordd. Instantly you know either you spelled it wrong or it’s a word MS didn’t put in its wordlist. In Excel, if you have a row of 10 numbers, and then create a sum to add 9 of those numbers, Excel will highlight the sum, and tell you that you missed a number. These tools work because MS programmed some basic knowledge about common usage into their program. But we can go further.

If you had a company with three divisions, Alpha, Bravo and Charlie, you might create a big spreadsheet with multiple pages, and showing info about various aspects of each division. Then one day you get a new division, Delta. You now need to go through a lot of work extending your spreadsheet in all its pages to include Delta. It could very easily happen that you miss a section, or that you add the section wrongly. And when your rapidly growing company adds Elephant and Foxtrot, the problem comes up again.

Knowledge-based software can help. We first create a tool which recognizes company division tables in spreadsheets. (This tool is highly reusable for all manner of spreadsheet analysis, so it’s not a waste. It’s also very easy software to write.) Then we tell the tool that all such tables should have 3 divisions (Alpha, Bravo, and Charlie). This is your knowledge base. The tool will instantly tell you that all tables are good. Now you change the knowledge base to mention Delta division. Immediately, the spreadsheet analysis tool knows all your tables are wrong. Using the tool you could quickly jump to each and every one so you can fix them. And the tool will tell you when you are done.

With a little more work, we can take this example a little further. Suppose we teach the tool about the consistent structure of these tables. About the formulas that need to be replicated. Again much of this teaching can be quite generic about spreadsheets in general, but some of it will be about your particular spreadsheets. Further, we instruct our tool on how to modify spreadsheets by adding rows. These are trivial functions. Finally, we tell the tool that upon request, it should modify all spreadsheet division tables according to the list of divisions. All the above programs are trivial and the program can be written quickly so that whenever you have a new division, you type its name, press a button, and bang! Every table in your spreadsheet has an extra line, and it’s correct. Now if it weren’t correct, our previous tool would detect faulty tables and tell you, and you’d know your updater has a bug.

Now for the magic: each of the simple bits of functionality programmed into our tool were about reading simple data, recognizing simple data, and making simple changes to simple data. In the above example all those items were programmed specifically for the company divisions spreadsheet tool. But all of the details can be abstracted away, and replaced by data specifying what to recognize, what to match, what to replace with what. And if we do that we end up with a tool that can be instructed on maintaining any spreadsheet – you just need to describe the patterns and formats.

And for one further level: A spreadsheet is just a data model. It can be described by bunches of objects with attributes. By abstracting away from the spreadsheet, we end up with a tool to match any data for any system. But so far, it only handles the kinds of patterns that a spreadsheet handles.

Aside: XML is a language which has been used for describing data, and is specialized for various forms of data. A spreadsheet XML format exists, and would work well for our purposes. Alternatively, APIs in the spreadsheet program can expose the cells and update mechanisms.

The program for describing spreadsheet patterns is quite simple. This is the case for most other commonly recognized patterns in many other applications. This is especially the case for precisely defined subjects where right and wrong ways of doing something are clearly specified. Those clear specs are exactly writable as knowledge for a computer to evaluate. Simple programs can be written to evaluate whether the data put in the computer (the CAD design, the source code, the process plan, the purchase order) meets the specifications.

Such simple programs enable evaluating all kinds of specifications, so that the engineer or accountant will know instantly what is not done, what is done wrong. There are limits though. Software can only evaluate what has been clearly told. It can only be told what the expert clearly understands, and fits the kinds of patterns the software has been programmed to recognize.

Nevertheless, a program that only knows some things can still be useful, because it will never miss on those things, and help you work better because you know longer need to pay attention to them.

There are several sites out there that help you to know what things are popular, starting with Google (PageRank is based on how many sites link to a page), and then Stumble, del.icio.us, and Digg. But to date I haven’t seen any site that really lets people compare things side by side.

What I’m imagining is something like PC magazine’s (actually, most computer mags) product comparison articles. In these articles, the authors try a number of related products, analyze the features, and then rate each product on those features. From these they produce a composite score. As a reader who is relatively ignorant of the tool being reviewed, I might decide that to investigate the tool scoring 9.7/10 overall. Meanwhile, a somewhat informed reader might decide that “indexed report speed” (I’m making up a random feature here) is the most critical feature and will opt to investigate the two products that ranked highest in that feature.

Meanwhile, over at BoardGameGeek.com, the operators have created a very impressive system for game lovers (and publishers) to add their favourite games, describe them, and break down the details. Once posted, others can add information, and can rate the games. Then they can construct lists of games (such as Alan’s favourites, Very Fast 2 Player Games, Axis and Allies Variants, etc.) People can browse these lists to find games related to ones they already like.

These concepts need to be combined. Moving beyond board games, we could use a site to compare webapps. How does gmail compare to yahoo mail to hotmail? How does blogger compare to the other blog systems? What sites work well with Google Maps? The list goes on and on.

So in this hypothetical system, you enter your favourite sites. Another person adds feature lists to some of these. Someone else makes lists which puts several comparable sites side by side. Everyone rates the sites, and rates the features of the sites. People make comments and discuss the sites. Then we can all see feature list comparisons, find related sites quickly, and know in short order to which site should turn to help get moving in the direction of web-based backup, or remote access to my home computer or whatever.

If such a site got going, thousands of people would organize, catalog, describe, rank, and compare thousands of sites on the net. And millions would be able to visit for a very quick rundown of what’s the best of what. Just like today they turn to PC Magazine for one comparison of 5 products, tomorrow they will turn to youcomparemillionsofproducts.com (or maybe the creators will come up with a better name) for instant comparisons of everything valuable under the sun.

You could search this database by name, tag, popularity (number of ratings), quality (avg number of stars), etc. And you will find what you want much faster than Google or Stumble could possibly achieve.

So, do you agree? Has this already been done and I missed it? If not, what will it take to get this going? Please share your opinions.

Alan

Lifehacks featured a thought that forms an interesting counterpoint to my last post. Try the link

I’m in the middle of reading this fascinating book, Wikinomics. It raises a lot of ideas about the phenomenon of mass collaboration, which of course gives me lots to discuss. As a demonstration of their concept, the authors created a wiki on their site and invited readers to create an extra chapter to the book.

This invitation had a measure of success, and they indeed got their chapter produced. It makes for an interesting read, and is packed with ideas and thoughts. Yet to my eyes it lacks the cohesion and clarity of thought represented in the book. The original wiki likewise exhibits the same phenomenon. Wikipedia on the other hand is relatively coherent. In order to achieve this they took advantage of a common understanding of the encyclopedia format, and made policies to guide the writers. Lots of standardized tags constantly advise readers how to improve pages, if they are so inclined.

But the real key to achieving quality is a dedicated core of editors constantly working to encourage and sometimes enforce the standards. This seems to be the the reality of wiki writing. It’s good for gathering ideas, organizing ideas, and developing ideas. A bunch of people can work on a site and develop information rapidly. But when it comes to cohesion, clarity and quality, the multitude requires editors to reign in their work and bring it to order.

Yet I don’t think the masses, even with editors, could have written the original book. Maybe I’m wrong, but it seems to me that there is something a small group or an individual can achieve that a group can’t. I can’t quite put my finger on it. Maybe it’s “art”. Not that the masses couldn’t create art, but that it wouldn’t be the same thing.

In 2005 at the University of Washington, Barbara Grosz of Harvard hosted a
colloquium on collaborative systems, where she talked about her research on getting computers to interact nicely with people.

As one example, she shows a program which works with a word processor to automatically lookup journal references when the operator types hints that a reference is needed. This is kind of software is relatively simple to construct (if you know some good sites to find the references), but can over time save researchers hours.

After this she discussed computer programs that work in a shared environment with people where each participant has its objective, and considers how programs can learn to cooperate with people by sharing resources to help both participants achieve their goals more efficiently than if they didn’t collaborate.

These examples are just a glimpse into where software will be going in the near future. Check it out!