MultiValue Trilogy: Overview

by Dawn M. Wolthuis

Copyright © 2002 Tincat Group, Inc.

This set of SOFACards provides an introduction to the MultiValue or Pick® industry. Use it to introduce technical professionals serving as managers, developers, marketing professionals, or students to the history and current state of the MultiValue industry. Follow this set by reading the Data and Commands sets to complete the trilogy.

There are so many stories of successes, litigation, mergers, theft, passion, brilliance, and even international intrigue that these cards are but an appetizer. The MultiValue legacy is sure to move forward - it is NOT just the past!

MV: MultiValue

The industry surrounding the Nelson-Pick data model platform.

Flexible, non-first normal form software environment supporting a multi-billion dollar industry with users numbering in the multi-millions. Call it what you will -- the industry's best kept secret, a techno-cult, a legacy system, or a niche technology for Pick® zealots, mid-size companies, or old-timers -- developer productivity and bang for the buck solutions are not yet out of fashion. The family tree of MultiValue database vendors is extensive, with current products including those from IBM®, Raining Data®, Northgate Information Solutions, Via Systems, Revelation Software, and jBASE®. The MultiValue term and corresponding logo were launched in 1996.

See intl-spectrum.com/aboutmv.htm

PICK®: Pick, It Continues Kicking

The MultiValue industry was and still is, commonly known as "Pick"

Named for Richard Pick, this ingenious data model, developed in the mid-1960's, provides a software environment where data is process and process is data. Text-based document handling is integral, given that source code "files" are each records in a Pick file (process as data). Querying a data element is just as easy whether the field is single or multi-valued, stored or computed (data as process) or even metadata in this world of untyped data hosting only variable-length structures. Flavors of MultiValue from the Microdata lineage avoided Pick's name or indicated they were "Pick-like" . A gaggle of lawyers and intriguing legal stories have been prompted by Pick.

See jes.com/cdp/cdp_faqx.html

MVVM: MultiValue Virtual Machine

Once an operating system, now a virtual machine on other OS's.

It's an operating system, it's a database, it's a query language, it's a programming environment & for most implementations of the MV data model, it's a virtual machine.* MultiValue BASIC compiles to MultiValue bytecode, aka p-code. This has permitted implementation on a wide variety of platforms over the past forty years. There is every reason to think this model will thrive in the coming decades as well, especially given its resemblance to XML data structures. VM here could also stand for Virtual Memory, reminding us that the MultiValue platform was revolutionary in including virtual memory management from the early days.

* jBASE® is a notable exception, compiling to C.

OLAP: On-Line Analytical Processing

Granular OLAP data in a standard RDBMS gets HUGE!

Query MV databases using SQL-92 tools and you will find that result sets can grow by leaps and bounds compared to MV Query tools. Store data warehouses in a MultiValue structure and both the storage and retrieval of data is more intuitive and compact. Store less summarized data by including computed elements, summarizing on the fly. Until mainstream tools understand multivalues by adhering to SQL-99 or XML standards, use an MV OLAP tool (www-3.ibm.com/software/ data/u2/mits)or warehouse data in an MVDB and extract data marts (databeacon.com) for OLAP.

See intl-spectrum.com/Publications/ SepOct2001.pdf p28 Diamond in the Mainstream, VanArsdale

XML-DB: eXtensible Markup Language DataBase

XML hosts a data model remarkably similar to the MV model.

XML-DB could refer to a database storing XML documents "in tact" but is eXtended here to refer to a DB used to manage XML documents. Will there be a revival in the Pick® world with the XML data model looking uncannily like MultiValue with delimited ascii text strings of variable length and embedded tables? In the 90's few people would admit to working with non-first normal form data, so the MultiValue stealth industry stayed below the radar. Today, the XML data model to some extent is the MV model. Is it time to uncloak ourselves and reveal that we have embedded lists in our variable-length data? Is there any better way to persist XML data than with a mature MVDB?

See w3.org

NF2: Non-First (NF) Normal Form (NF)

Instead of MultiValue, Pick®, Post-relational, or N1NF, call it NF2

The MultiValue database model is NOT in 1st normal form, according to E.F. Codd's 1974 principles. Sometimes called "Nested Relational," MV elements can themselves be lists of elements. Codd does include such structures in his 1970 ACM paper. SQL-99 and implementations such as PostgreSQL, as well as Oracle® now permit NF2 data too. So what!? Unfortunately, ODBC uses SQL-92 and does not address tables with hierarchical relationships, which is one way to view multivalues - parent tables related to child tables. SQL-99 and the emerging XQuery standard DO include support for NF2 structures. Progress!

See www-3.ibm.com/software/data/u2/pubs/whitepapers/nested_rdbms.pdf

U2: IBM UniVerse® and UniData®

IBM has the largest installed MultiValue database licenses with U2.

Big Blue was there at the first customer installation (U.S. Army, 1969) of what is now known as the MultiValue environment. IBM Heidelberg did research on nested data structures in the 1980's. Now supporting well over half the MultiValue Value-Added Resellers, customers, and end-users, IBM is even more integral to the future success of this industry. UniVerse and UniData were acquired from Informix, who had acquired them with the purchase of Ardent Software, which was formed by the merger of Unidata and VMark. IBM has the rights to distribute U2 in Asia, with Pacific Unidata no longer in the picture. It is also the resting place for PI/Open (from Pr1me).

See www-3.ibm.com/software/data/u2

DP: Dick Pick (1938-1994)

Colorful pioneer of the MultiValue environment, aka Pick®.

Stories abound, even if sticking to the business side of life. A Berkeley-trained physicist, Richard Pick surely influenced the history of DP, i.e. Data Processing, while often missing from the historical accounts. A man of vision and passion, he was influential in his unique way. Along with a passion for his software he seemed also to understand those partial to "wine, women, and song" (or "beer, babes, and boats" ?). He was a programmer first, with unconventional marketing, management, and sales approaches. Dick windsurfed, litigated, spun technology insults, and lived on the edge; and the edge of the industry is right where you will still find his devotees.

See jes.com/cdp/cdp_faqx.html

MVDB: MultiValue DataBase

A database with MultiValues based on the Nelson-Pick data model.

Typical relational databases cannot match the productivity of the MultiValue database, sometimes called "Post-relational," simply by permitting lists within data elements. They would also need features such as variable length data structures, untyped elements, user-defined vocabularies and custom functions specified as metadata. MVDBs require minimal database administration, resulting in a low cost of ownership. They are often bundled with application software and sold by Value Added Resellers. You might be surprised by the portfolio of companies running MVDB's, some of whom might also be surprised to find this lean and mean database on site.

See multivaluedatabases.com

GIRLS: Generalized Information Retrieval Language & System

The specified language at the start of the MultiValue industry.

GIRLS started it all, beginning with designs related to people and output, rather than input, storage, or processing of data in the early 60's. Don Nelson joined TRW in 1964 and drafted detailed flow charts when the U.S. Army contracted with them to put query tools in the hands of soldiers. GIRLS took shape so soldiers could ask questions in English to make decisions in the field. The query languages that evolved from GIRLS are easier to use than SQL and pre-date it, with the Army's installation in 1969. Sequel was developed in the mid-70's based on the concepts from Codd's relational work that began in 1969, with resulting SQL standards introduced in 1986.

See jes.com/cdp/cdp_faqx.html

RDBMS: Relational DataBase Management System

A set of tools used to manage data in relations, a.k.a. tables.

Flip through a textbook on databases and you will find descriptions of Network/Codasyl, Relational, Hierarchical, and perhaps Object and Object-Relational models to add to flat sequential and indexed files. Pick® is rarely included. Some refer to MultiValue as Relational or Post-relational, but on a timeline it is Pre-relational. Relational is the most popular database, conforming to the 12 principles set forth by E.F. Codd. The non-1st normal form of MultiValue is not the only reason it doesn't make lists of database models. Database theory stems more from computer handling of 80-column cards, where MultiValue arose as a language for people to use. SQL is a second language for MultiValue.

SMA: Spectrum Manufacturing Association

A consortium formed by manufacturers of Pick® software.

Formed in 1985, this group discussed standards, bringing such flavors as Prime Information more in line with R83 Pick environments by referencing the Pick Pocket Guide. In spite of efforts to get more collaboration among the various database software vendors, each company went their own way for such extensions to Pick as GUI, web, SQL, JDBC, and other required upgrades. Although the SMA is no longer, International Spectrum from IDBMA, Inc. holds annual international and regional conferences and also publishes a magazine dedicated to the MultiValue industry. They led the charge to get the MultiValue name and logo in the 90's.

See intl-spectrum.com

BASIC: Beginner's All-purpose Symbolic Instruction Code

Dartmouth BASIC with easy data access, sprinkled with Pascal.

Dartmouth College professors Kemeny and Kurtz wrote the language BASIC in 1964 with features taken from Fortran and ALGOL. They introduced interactive computing with the INPUT command. Pick adopted this 3GL and Ken Simms wrote a compiler in the mid-70's with significant enhancements for database manipulation. Concurrent with UCSC work on Wirth's Pascal p-machine, it was written at University of California, Irvine (UCI) to compile to p-code. Simms completed it with Pick in Hawaii, and it shows. The clearly superior DataBASIC result is not to be confused with Gates/Allen BASIC, also mid-70's. ANSI BASIC '88 - a day late and a dollar short, perhaps?

See jes.com/pb/pbapg.pdf

GIM: Generalized Information Management

The first installation of the Pick® model to the Army was GIM.

In 1965, the U.S. Army contracted with the TRW company, with Don Nelson starting the effort. Richard Pick then came on board. Although not a computing expert by background, Nelson drafted flow charts for the GIRLS language, now purposed to give soldiers with no computing background the ability to make sense of a database. The resulting system, GIM, including COBOL components for updates, was implemented in 1969 to track Cheyenne helicopter parts during the Viet Nam war. Subsequent versions were in use as late as the 80's for tracking KGB agents located in the U.S. If any derivative of GIM is still in use by the military today, I can't say.

PU: Pacific Unidata

PU litigated its right to distribute UniData in Asia during the 1990's.

Yue Ming came to the US and studied databases, writing a paper about the Non-First Normal Form model. The story is that he was led in a dream to implement this model in UNIX and coincident with this a Prime Information VAR in Denver was led by the American dream to port from Primos to Unix in the 80's. They teamed up and this implementation of the model-not-to-be-identified-as-Pick® became UniData® and looked like a UNIX version of the popular Prime Information. After serving as President of Unidata, Yue Ming sold his interest and returned to Hong Kong, launching Pacific Unidata, which signed an agreement in 1994 with Unidata for rights to distribute UniData in Asia. Litigation followed.

AVON: Asian Violations Over Nothing

Bummer to be sued for using a database you licensed.

Continuing in the litigious tradition of the Pick® world, in 1996 Pacific Unidata sued Avon for using the UniData database in their joint venture in China. In 1995, Guangzhou Avon had licensed UniData along with application software from a UniData reseller. Pacific Unidata held the copyright for UniData in China. Avon paid a fine from the first law suit prior to being hit with another. Later that year, PU sued Unidata and its then president, the former partner of the PU president. The set of PU lawsuits has been put to rest, with IBM now holding the rights to sell UniData in Asia. PU vs Avon is notable as a case where a licensee is internationally liable for knowing the ownership of its licensed software.

µData: Microdata Corporation

Microdata's Reality -- the first commercial installation of MultiValue.

After Pick left TRW in 1969, after he partnered with Microdata and started Pick & Associates, after Intertechnique delivered boards to interpret the REAL language, but before Don Fuller of Microdata kicked Dick Pick out of bed, Microdata shipped its first Reality system. They tore up the strip with a first-of-its-kind dealer network. After McDonnell Douglas Corporation acquired Microdata in 1979, after it was renamed McDonnell Douglas Computer Systems Company, after McDonnell Douglas Information Systems International was spun out, and after "Douglas" was dropped from MDIS, it is now Northgate Information Solutions. Reality is Reality, now at V9.1.

See northgate-is.com

PI: Prime Information

First implementation riding atop another Operating System, Primos.

The first commercial implementations of Pick® were as the native operating system for various hardware platforms, starting with Microdata - Intertechnique. By the mid-80's, Pick Systems® had sold licenses to more than 30 hardware vendors who used it as a native OS, based on R83 standards. In the late 70's, Devcom, a Microdata VAR, emulated the system on Pr1me Computer's Primos OS, an OS written in Fortran (which was cool ? cause this then college girl could readily modify it). The resulting Prime Information was so successful that Pr1me bought it. It influenced other ports, including uniVerse, UniData, and Revelation. PI and Pr1me are now history, sigh.

See malch.com/prime/primefaq.htm

P-code: Pseudo-object code

This bytecode for the MultiValue Virtual Machine, the p-machine.

After Wirth specified the Pascal language in the late 60's, the first Pascal compilers were developed at UCSC in the '70s. This included compiling Pascal to p-code for a pseudo-machine, making it portable. Such a p-machine was running on a Xerox Sigma-7 at UCI where Ken Simms worked on the DataBASIC language for maintaining Pick's system. Most DataBASIC flavors today compile to p-code rather than object code. The p-code, functioning like Java"! bytecode, runs in a Virtual Machine. Exceptions are jBASE®, which translates to C and then uses a C compiler and the MaVerick Open Source project, compiling DataBASIC to Java bytecode instead of p-code.

See maverick-dbms.org

RD: Raining Data®

What do you get when you cross Pick® with Omnis®?

With µData now evolved into Northgate Information Solutions and the PI branch of the MultiValue family tree now dominated by IBM®, the R83 or "true Pick" branch is now mainly consolidated in Raining Data Corporation, headquartered in Irvine, CA. Pick Systems®, tracing back to Pick & Associates in 1972, was bought out of probate in March 2000. They acquired the MV systems of General Automation, who had already acquired many of the Native Pick operating systems. Pick Systems merged with Omnis Software to form Raining Data in December 2000. The "multidimensional database management products" from Raining Data include D3, mvBase, and mvEnterprise.

See rainingdata.com

LIST

Corresponds to the SQL SELECT statement for database queries.

Influenced by the vision of Grace Hopper using English-like statements in the '58 FLOW-MATIC, and by subsequent 60's COBOL compilers, Nelson, Pick, and Andrews wrote in their TRW 1967 paper that their new system must "accommodate natural language queries and multiple technical vocabularies." They gave this example:
LIST THE TITLE AUTHOR AND ABSTRACT OF EVERY TRANSPORTATION SYSTEM REFERENCE WITH THE PRINCIPAL CITY "LOS ANGELES" AND THE GROUND EFFECT MACHINES "AVC-1" OR "HOVERCRAFT1

See rainingdata.com/company/history

1 GIM-1, a generalized information management language and computer system, American Federation of Infromation Processing Societies, New York, NY 10017. Volume 30, 1967, p. 169-172

EEA: Economic Espionage Act

What does MultiValue have to do with economic espionage?

The history of the MultiValue industry provides more entertainment than All My Children and a Grisham book combined. In addition to law suits such as those related to rights to use the Pick® data model, court proceedings following Pick's death, and the Pacific Unidata cases, is the following story. The EEA was enacted in the US in 1996 based in a large part on a 1994 incident resulting in both an employee and a former employee of a MultiValue software company being accused of corporate espionage. Word on the street is that it was an act of Chinese government espionage, but that was not proven. MV heritage includes spies!

See rf-web.tamu.edu/files/SECGUIDE/Spystory/Ellery.htm

Delightful SQL

While often considered pragmatic, SQL is not known as delightful.

This tagline for Tincat Group, Inc. might give the impression that SQL is a delightful language. Not so. But there COULD BE a delightful sequel to SQL. The MultiValue query languages are easier to use for many complex tasks. Known in flavors such as ENGLISH, FRENCH, ACCESS, RECALL, retrieVe, INFORM, R/LIST, Info/Access, AQL, jQL, and UniQuery, this language could handle queries of text in XML documents. If XQuery, a SQL-99 implementation, or another standard emerges as a query language of choice for XML, it should also be an adequate new standard for queries against Pick® data structures, provided it is just as delightful as ENGLISH.

See www.w3.org/XML/Query

MultiValue Branches

R83, µData, and PI

These cards use the terms R83, µData, and PI to identify three not-entirely-distinct streams of development of the MultiValue platform. The MultiValue Family Tree diagram, available as a free pdf from store.tincat-group.com, provides a corresponding visual. These terms are used at times in the cards as shorthand identifiers for the three interwoven but separate branches of MultiValue.

In the late 70? s, the Pr1me Information (PI) stream started, emulating functionality roughly equivalent to Microdata? s Reality 3.x product (µData), which was roughly equivalent to Pick® R77. After that (e.g. R83), the streams diverged. Pick historians indicate that R80 is the software version that was marketed as R83, but the marketing term serves our purposes in these cards none-the-less.

Terms related to MV data and commands are identified with the two primary flavors -- R83 and PI -- with µData typically aligned with the R83 camp.

SOFACards MultiValue Overview Pack

Many people and web sites were consulted in the development of these cards. Special thanks to the following people whose help was invaluable and whose years of experience in the MultiValue world total over 80:

• Dave Weaver for prompt responses of encouragement when I needed it and excellent proof-reading skills in both form and content.
• Mark Pick for sharing stories and sending the thumbs up signals.
• Jon Sisk for knowing so much and coming to my aid in the 11th hour, plus his detailed publications that aided my research.
• Virginia VanAndel for "doing lunch" and passing along helpful intuitive reactions as well as knowledge from her vast experience.