Smarter tools: An introduction

Computers are great tools which haven’t been fully exploited yet. So far software generally provides tools to enable you to tell the computer what you want, and the computer gives it to you. Word or Excel etc are gen purp apps that give you tools. SimplyAccounting and SolidWorks are more vertical market tools that permit more specialized operations, but they really don’t know what you are up to. QuickTax is a step better. Because tax forms are so standardized and the rules apply to everyone, software companies are able to create software which will figure out your taxes for you, and quickly. Because it knows the rules of tax declarations.

Knowledge is the future of software. If you could explain to a computer about the kind of work you do, tell it what your expectations are for the results, then you have a starting point. From that description we can start creating programs to help you get your work done. A computer program can “look over your shoulder” and tell you when you make a mistake. MS Word does this when you type something that isn’t a wordd. Instantly you know either you spelled it wrong or it’s a word MS didn’t put in its wordlist. In Excel, if you have a row of 10 numbers, and then create a sum to add 9 of those numbers, Excel will highlight the sum, and tell you that you missed a number. These tools work because MS programmed some basic knowledge about common usage into their program. But we can go further.

If you had a company with three divisions, Alpha, Bravo and Charlie, you might create a big spreadsheet with multiple pages, and showing info about various aspects of each division. Then one day you get a new division, Delta. You now need to go through a lot of work extending your spreadsheet in all its pages to include Delta. It could very easily happen that you miss a section, or that you add the section wrongly. And when your rapidly growing company adds Elephant and Foxtrot, the problem comes up again.

Knowledge-based software can help. We first create a tool which recognizes company division tables in spreadsheets. (This tool is highly reusable for all manner of spreadsheet analysis, so it’s not a waste. It’s also very easy software to write.) Then we tell the tool that all such tables should have 3 divisions (Alpha, Bravo, and Charlie). This is your knowledge base. The tool will instantly tell you that all tables are good. Now you change the knowledge base to mention Delta division. Immediately, the spreadsheet analysis tool knows all your tables are wrong. Using the tool you could quickly jump to each and every one so you can fix them. And the tool will tell you when you are done.

With a little more work, we can take this example a little further. Suppose we teach the tool about the consistent structure of these tables. About the formulas that need to be replicated. Again much of this teaching can be quite generic about spreadsheets in general, but some of it will be about your particular spreadsheets. Further, we instruct our tool on how to modify spreadsheets by adding rows. These are trivial functions. Finally, we tell the tool that upon request, it should modify all spreadsheet division tables according to the list of divisions. All the above programs are trivial and the program can be written quickly so that whenever you have a new division, you type its name, press a button, and bang! Every table in your spreadsheet has an extra line, and it’s correct. Now if it weren’t correct, our previous tool would detect faulty tables and tell you, and you’d know your updater has a bug.

Now for the magic: each of the simple bits of functionality programmed into our tool were about reading simple data, recognizing simple data, and making simple changes to simple data. In the above example all those items were programmed specifically for the company divisions spreadsheet tool. But all of the details can be abstracted away, and replaced by data specifying what to recognize, what to match, what to replace with what. And if we do that we end up with a tool that can be instructed on maintaining any spreadsheet – you just need to describe the patterns and formats.

And for one further level: A spreadsheet is just a data model. It can be described by bunches of objects with attributes. By abstracting away from the spreadsheet, we end up with a tool to match any data for any system. But so far, it only handles the kinds of patterns that a spreadsheet handles.

Aside: XML is a language which has been used for describing data, and is specialized for various forms of data. A spreadsheet XML format exists, and would work well for our purposes. Alternatively, APIs in the spreadsheet program can expose the cells and update mechanisms.

The program for describing spreadsheet patterns is quite simple. This is the case for most other commonly recognized patterns in many other applications. This is especially the case for precisely defined subjects where right and wrong ways of doing something are clearly specified. Those clear specs are exactly writable as knowledge for a computer to evaluate. Simple programs can be written to evaluate whether the data put in the computer (the CAD design, the source code, the process plan, the purchase order) meets the specifications.

Such simple programs enable evaluating all kinds of specifications, so that the engineer or accountant will know instantly what is not done, what is done wrong. There are limits though. Software can only evaluate what has been clearly told. It can only be told what the expert clearly understands, and fits the kinds of patterns the software has been programmed to recognize.

Nevertheless, a program that only knows some things can still be useful, because it will never miss on those things, and help you work better because you know longer need to pay attention to them.