End data duplication

EXECUTIVE SUMMARY: All of today's accounting and business systems are based on duplication of balance and transaction information in the separate books of each participant.  The resulting costs are not a simple doubling of labor, but a geometric function caused by errors and differences.  

The question is whether to improve the communication between accounting systems to keep their redundant data in agreement, or eliminate certain aspects of the duplication in the first place, by using a master record in a shared location.

Business transactions are always owned by two or more parties.  Any party may store these data on secure hosts on the internet, visible only to the parties to the  transaction.   Accounting systems should consist primarily of indexes or pointers to all the encrypted rows we own, on various hosts, and the encryption keys to read them. The elimination of data duplication would achieve billions in cost savings, surrendering nothing in privacy.  

The OMG has issued an RFP for global reconciliation of AR and AP systems. http://www.omg.org/techprocess/meetings/schedule/AR_AP_Facility_RFP.html .


The internet economy being designed and built today is based on redundant storage of data in multiple locations. Web developers are needlessly re-creating the same accounting problems as LAN-bound systems, and paper systems before them.

The availability of ubiquitous networking enables a fresh look at the designs underlying all accounting, sales, and settlement systems in use today.

Start with the basics: all businesses (and consumers) need the following obvious capabilities;

  • to send and receive payments over the internet,
  • to send, and receive, orders for products and services,  and,
  • to send and receive invoices.

Solutions are emerging for all of these, but there is no coherent, overall integration to get the bookkeeping done:

  • update our accounts receivable/payable for a sale or payment,
  • update our inventory counts for particular products, and
  • update our book balances of cash in bank, etc.

The common belief, and knee-jerk reaction, among e-commerce developers is that a nearly-utopian business environment will happen soon, as XML is widely adopted, and standards emerge:

  • standard XML formats for invoices, orders and payments,
  • widely accepted protocols for computers to link and process these files, and
  • encryption and legal infrastructure making them reliable and nonrepudiable, etc.

The next morning after these standards exist, the market will wake up with a hangover, and realize we need a lot of global directories or cross-references, for

  • customer/vendor IDs,
  • product IDs,
  • namespaces, industry markets, market segments, etc.

For example, what's the use of an invoice received thru the internet, having perfectly brilliant XML formatting, if the product IDs are not the same as my self-generated inventory IDs and vendor IDs?

Today's vertical portals and lock-in vendors are not exactly working overtime to fix these problems. Instead, they are building "communities" - with themselves as rent collectors. But eventually, horizontal integration will fix this problem. For five years we'll be mapping our codes against master codes, then we will use the same native codes.

But ten years from now, after we get all this apparatus working it will still require just as much complicated manual bookkeeping to maintain, because everything being designed today based on multiple, redundant data stores. We are just re-creating in e-commerce, the same data architecture we had in EDI, LANs, and manually before that.

That's sad, and unnecessary.

Fact is, there are two owners to every transaction in the universe: the buyer and seller. The master, and only, record of that transaction should be stored securely on the internet, where those two parties can always reach it.

For those transactions, both parties have also agreed on the customerID, vendorID, and product IDs. Accordingly, the record of the customerID, vendorID, and product IDs they agreed upon at time of transaction, also belongs in encrypted rows, on the internet.

Accounting systems should consist only of indexes or pointers to all the encrypted rows we own, on diverse remote hosts and public transaction repositories and webledger sites on the internet, and the encryption keys to read them.

Yes, just indexes.

  • Perhaps with replicated data locally, for faster reporting,
  • Perhaps with additional value-added information, locally,
  • Perhaps even with traditional paper hardcopies.

But the true, authentic data would be on the hosts.

This would not surrender one bit of privacy, but it would achieve a quantum leap in efficiency.

Maintaining double entry accounting in both buyer and seller locations is a redundant data structure, if viewed globally. The mirror image of every sale recorded into receivables by a seller, is also recorded as a purchase, into accounts payable by a buyer.

That's quadruple entry. Since every transaction in the developed world also causes quadruple entries when it clears thru a bank,  that's octuple entry. For a graphic tutorial of this, see http://www.gldialtone.com/Exploration.htm

Whenever data is stored in more than one place, these consequences always result:

  • The redundant data stores attract updates from different classes of users.
  • The data stores are updated at different times, and in different levels of aggregation.
  • Neither data store is ever correct or timely.
  • Time is wasted confirming and reconciling one against the other.
  • Costs arise due to business errors based on wrong information (overdrafts, credit/collection errors, etc.)
  • Systems are developed to address the problems, by automating the checking and comparison of the two lists, or to post all changes to both lists, etc.
  • Those secondary systems introduce rigidity and problems throughout the software environment.

These costs and consequences are geometric in nature, and are the root of our current employment of tens of millions of clerical workers, each operating completely isolated accounting systems.

A shared transaction repository can deliver self-reconciliation, linearly, with each new use.  This is not a massive chicken/egg problem.  It does not require an impossible leap to universal adoption, to begin yielding benefits.  The benefits would be accrued by any accounting system having the most rudimentary interface to the internet such as an SMTP inbox and outbox. These capabilities are already present within many existing platforms and trading relationships (trading hubs and marketplaces, portals and purchasing dashboards, private exchanges and supply chain etc.)

Subledgers having this capability can be implemented incrementally, alongside existing legacy accounting software. Some approaches to "merging" multiple transaction sets are

  • Tightly coupled interfaces DCOM, CORBA, or RMI (including database replication)
  • Loosely coupled document interfaces such as XML and EDI, and
  • Accounting consolidation models, e.g. summary journal entries

The consolidation of sub-ledgers of departments and subsidiaries is a conceptual model and methodology which is well-understood by millions of accountants and accounting software developers.

Accounting consolidation is a comprehensive, robust methodology that provides ready-made solutions for merging multiple transaction sets of arbitrary complexity.  Any global component architecture that allows sovereign web applications will require solutions that merge the transactions executed on diverse platforms.  E-commerce developers will encounter many challenges in fractional shares, derivative instruments, eliminations, and reconciliations of all kinds of intercompany balances.   These developers are only beginning a journey into these business processes.

Sooner or later, all these systems will come home to daddy: consolidation accounting.  Subledgers will consolidate into the root ledger of the enterprise.

E-commerce architectures will plug and play with existing LAN-bound software, if designed from the outset to deliver output in consolidation structures. Several families of XML schema already exist for General Ledger transactions and trial balances. http://www.arapxml.net/index.htm  View the powerpoint slides.

Let's build something worthwhile. Let's fix the problem, this time around.

Here are some architecture ideas, for moving G/Ls and transaction journals to shared hosts on internet. A shared transaction repository and some simple choreography. http://www.GLDialtone.com/STR.htm 
http://www.GLDialtone.com/fbc.htm 

It is my belief that a public transaction repository could also be built, today, running wholly on the basis of existing XML schema, with the context, negotiation, and choreography of ebXML.

* Todd F. Boyle CPA http://www.GLdialtone.com  1999