Retro Standards - Part 1: File Descriptors

Retro Standards!

This is the first in what is hopefully a small series (let’s see how lazy I am!) regarding various retro/vintage computing “standards”. I put standards in quotes as many of the formats and methods that will be discussed are often very informal.

For the first installment, I want to talk about file descriptors that have been (and in some cases still are) used throughout the pre-Internet age Bulletin Board System (BBS) communities, art and warez scenes, and so on.

The Pre-Description Era

Almost from the time they existed, BBSes have been a place to not only exchange messages and chat, but to share files. In simpler times a (short!) filename such as TREK.BAS or ALIENS.LBR was often enough to convey what a particular file or archive was all about. If it wasn’t, the set of files in your immediate community was generally small enough that message bases (BBS era forums) and chatting with your friends was usually enough to lead you to the goods: “Hey, you should really check out TREK.BAS!”. As things progressed, systems would allow short user input descriptions for uploads. Of course this resulted in a mixed bag of quality and a single line description is hardly enough to describe how awesome the latest upload was!

A file listing from Captain’s Quarters BBS.

Directory Databases

With file trading becoming more and more popular on Bulletin Board Systems, and the sheer number of files to choose from, something had to be done. Various systems and file formats started to show up allowing users and SysOps to add descriptions to listings of files. Systems came about for transferring these flat file “databases” between boards, importing them into other tools, and converting between formats. While this may not sound like much at first, think of it this way: Our ambiguous TREK.BAS could also now come with a description – “Star Trek: Navigate the USS Enterprise!“.

FILES.BBS

A very early and popular “standard” for file descriptions was the FILES.BBS format. Generally the file is composed lines containing a filename, a space or a tab, then a description (Remember: Early filenames were short and could not contain spaces on most file / operating systems).

TREK.BAS Star Trek: Navigate the USS Enterprise
ALIENS.LBR A Space Invaders like game.

At this point it’s hard to say exactly who or what software used this format first. A very early reference can be found in Fido 10 from 1985. The archive contains a FILES.BBS using a format that seems to support - prefixed lines for comments, and ends with a number of 0x1a (SUB) characters. What became the standard is much more like the very simple example above.

FILES.BBS is then mentioned in the December 1st 1986 FidoNews. Opus-CBCS was also using the format around this time as seen in OSTRUCT.ZIP. The way the format is discussed in relation to FidoNet makes me think it was a precursor to TIC file Desc and LDesc fields. For example, see
FwdSpec - A Collection of Notes on Moving Files in FidoNet (1988).

The format is wildly successful in the BBS world. Many systems start to utilize FILES.BBS. For example QuickBBS v1.02 makes mention in QUICKBBS.DOC found within QBBS102.ZIP.

Fun fact: the aformentioned archive cannot be processed with most modern ZIP tools due to lack of “UnReduce” support that may not be included due to licensing restrictions. If you want to take a look for yourself, build InfoZipunzip with the support by including unreduce.c and defining USE_SMITH_CODE whilst undefining USE_COPYRIGHT_CLEAN.

Around 1991 Opus v1.70 is released and moves to a newer format though many tools can convert between the two. Systems still being written/maintained today such as Synchronet continue to make use of this file.

4DOS DESCRIPT.ION

Sometime about 1989 a company called JP Software released a program called 4DOS. One of the features it brought to the table was a file called DESCRIPT.ION. What this did was allow users of 4DOS to manage description of files in a similar fashion to FILE.BBS via a utility called 4EDIT.

The format is quite simple. From descfile.txt:

filename.ext Description[*<ID>Other program info]...<CR><LF>

Where ID is a single byte registered software identifier (I guess they didn’t expect many implementations…). In it’s most basic form a DESCRIPT.ION isn’t any different than a FILES.BBS file and may look similar to the following:

SNAKE.BAS A snake game in BASIC
ZORK1.ARC Zork I - The Great Underground Empire

(4EDIT v3 working with some sweet warez!)

Perhaps a advantage of DESCRIPT.ION over FILES.BBS is the “Other program info” area of the format. Developers could store additional metadata in this space in theory. Authors created utilities to convert between the two formats as well. Ultimately variants of the format came about as well. For example quotes around the filename allowed LFN’s containing spaces:

"Microsoft Plus!.zip" Microsoft Plus! for Windows '95

…of course all of this still left much to be desired. A major drawback of these types of listings being that the user or operator still generally have provide a description and there is no way to pre-package a description in archives.

Archive Descriptions

As file trading became more popular (not to mention complex and with larger files) the status quo was to use archives. That is: a collection of often compressed files within a single file. This alone leaves the problem of file descriptions. So what do you do? Add a descriptor file to the archive! This simple idea took many forms…

“FOR” Files

As pointed out by by fsxNet user:

As I’m playing with CP/M software a lot, I found that many programs here

come bundled with “*.FOR” files (eg. QTERM.FOR for the program “QTerm”).

These files also contain a short information about the specified software.

But I don’t know anything about standards or something like that…

It seems early archives found within the CP/M world often contained .FOR files with short summaries of the archive. Some examples can be found within the Tesseract RCPM+ Public Domain Software archive. Upon inspection, it is hard to determine a format other than they appear to wrap before 80 characters in length. Annoyingly .FOR is also the extention used by FORTRAN source that was also popular at the time making searching for such files a bit of a pain.

FILE_ID.DIZ

If you’re reading this far, you might very well already be familiar with this “famous” description file. The FILE****IDentification Description In Zip or FILE_ID.DIZ originated from a tool named “PCBDescribe” by Michael Leavitt, a support technician at Clark Development Corporation who released the specification and tool to the public domain. This was picked up as a standard format in PCBoard itself soon thereafter. From there an individual by the name of Richard Hollar seems to have pushed it as standard – something the Association of Software Professionals picked up on (which was a fairly big deal at the time).

It’s currently unclear exactly when the format first saw light, however we can gather some hints…

PCBDescribe from 1991:

PCBDESC.ZIP     28261  10-02-91  PCBDescribe Version 1.11 10/02/91 release. A
                               | utility for PCBoard 14.5A systems that
                               | automatically detects description files
                               | included in uploaded ZIP files, and uses the
                               | contents as the upload description.  Also
                               | optionally appends the newest file date in
                               | the ZIP file to the description.

(From https://textfiles.habd.as/bbs/FILELISTS/southexp.lst.html)

Meta moment: Notice something about the listing above? Another variation of FILES.BBS/similar that a number of systems adopted allowing additional file information. Additionally multi-line descriptions are continued using the | pipe character!

From interviews gathered for The BBS Documentary:

Clark Development corporation pioneered the filename FILE_ID.DIZ to include description information in a file archive. Originally created to accompany their PCBDescribe Utility, the idea behind the standardized filename was to provide an expected place for the file’s description, removing the need of uploaders to manually type it in each time they sent the file to a new BBS.

We also see mentions in the next major PCBoard releases v14.5 demo disk archives (‘92):

PCBDESC will  accept descriptions contained  in FILE_ID.DIZ files
found in the ZIP being tested.  The FILE_ID.DIZ file  is an ASCII
text file, and can contain up  to 10 lines of 45 characters each.
The first  line of this file is the program name and version, and
the following lines describe the function of the program.

ATTENTION!   The  FILE_ID.DIZ file  is intended  for the  program
author's use in providing a coherent description of his  program.
In this  way, the author  and the sysop  can be assured  that the
program will be  properly described when  uploaded to a BBS.   DO
NOT use this file for BBS advertising - such use  is in violation
of the copyright associated with the FILE_ID.DIZ file

Third party tools such as the AUTODESC tool by Karl Schneider also start to come out and make mentions in their documentation:

11/24/91. MAJOR CHANGE! Added code to make Desqview Aware, CHANGED
DESCRIPTION FILE NAME. WC3DESC is NOT now supported! AUTODESC now
looks for either DESC.SDI or FILE_ID.DIZ for its description! This
has been requested by several people. Please erase any versions
of AUTODESC or PUTDESC dated before 11/24/91.

(Notice it also references DESC.SDI and a very obscure WC3DESC!)

TL;DR: Sometime around 1990-1991 we end up with a FILE_ID.DIZ format that can be embedded in archives to give them a much more proper description: up to 10 lines of 45 characters each.

By 1994 Richard Hollar has released a more formal version 1.9 of the “spec” and an example:

MY PROGRAM v1.23 <ASP> - A program which will
do anything for anybody. Will run in only 2k
of memory. Can be run from the command line,
or installed as a TSR. Completely menu-
driven. Version 1.23 reduces the previous 4k
memory requirements, and adds an enhanced
graphical user interface. Also, MY PROGRAM
now contains Windows and DESQview support.
Coming soon - an OS/2 version.
From Do-It-All Software, Inc. $15.00

The spec remains to say no more than 10 lines of 45 characters each, and attempts to establish some rules:

It should NOT contain any blank
lines, any form of centering or formatting, or any Hi-ASCII or ANSI

characters. (i.e. it should ONLY contain alpha & numeric characters)

And let’s not forget the dire warnings in previous excerpts! Well… I guess rules are meant to be broken!

As quickly as it was conceived the FILE_ID.DIZ format was adopted as the defacto standard by the underground HPAVC (Hacking Phreaking Anarchy Virus) and warez scenes where it continues to live on to this day. And while the 45 character per line limit has mostly stuck, the the rest of the rules were tossed in the trash. It became very common to see much longer descriptions and something else that is almost a must in the scene: ASCII art.

The Amiga scene also picked up on the format bringing a fresh aSCII sTYLE:

Here we see a release of Lemmings II by Fairlight rendered in the Topaz+ font. Notice the first archive has a almost pure ASCII description while the subsequent archives have a one liner. Perfect for listing on a 0 day board!

While the distribution of software has certainly changed since the age of BBSes, modern day FILE_ID.DIZ can still be found in freeware/open source archives as well as from the scene (though the latter especially has become much less common).

Plain ASCII is great and all, but sometimes you want some color! FILE_ID.DIZ authors have been known to use various color codes in the form of PCBoard, pipe codes, or ANSI escape sequences. Not all software supports all (or any) of the various forms however. The image below shows PCBoard @X## style codes in use on a Daydream system:

Incorporating ANSI escape sequences has become a popular way to colorize artscene related FILE_ID.DIZ files. Sometimes this comes in the form of such as FILE_ID.DIZ.ANS or just FILE_ID.ANS allowing compatibility with extractors that aren’t aware of the new filenames.

(Shameless plug: A recent Blocktronics release as seen on Xibalba BBS running ENiGMA½)

Collys

While a FILE_ID.DIZ works great for archives such as ZIP files, what about text files? A interesting solution to this problem mostly seen in the Amiga seen where collys were very popular was the advent of begin/end blocks for embedded descriptions. This allowed text files to be uploaded to a system and their description still extracted by a board by looking for the @BEGIN_FILE_ID.DIZ and @END_FILE_ID.DIZ markers (generally at the top of the file). This is another area in which I’m fairly fuzzy on the history thereof, so if you have such knowledge please drop me a line!

DESC.SDI

What seems to be around the same time frame as FILE_ID.DIZ‘s emergence is the DESC.SDI format. In fact, for a number of years it was common for various utilities to place both files in archives and for file processors to look for both. It’s not fully clear to me exactly when the DESC.SDI format came into existence or in which software it first appeared. The earliest I can confirm is from this file from 1990 containing the file.

The format itself is simply a single line description no more than 60 characters in length. The character limit can befound referenced in the documetnation of HSLTAG v1.02Beta (1993):

S = DESC.SDI description format is a single line description of
a file 60 chars long. HSLTAG will offer you a single line
editor to enter your description if no internal DESC.SDI is
found in the file.

As well as the Wildcat! v4 Sysops Guide (1995) where FILE_ID.DIZ is also mentioned:

DESC.SDI is a single line description, up to 60 characters in width.
FILE_ID.DIZ is a multi·line description, up to 40 characters in width, and as
many as 15 lines deep. wcFILE can extract these two files from ZIP and
LZH archives, and import the information automatically.

While DESC.SDI seemed to have a good run, FILE_ID.DIZ is the format that ulitimately survived.

WC3DESC

A very short lived description attempt seems to have both shown up and disappeared in Mustang Software’s Wildcat! versions 3.x called WC3DESC. Blips of information on this format can be found within Wildcat! related utilities and archives containing them are scarce.

Portable Application Description

A newer format that was designed to replace FILE_ID.DIZ was Portable Application Description (PAD). While it may have ended up in some ASP approved shareware distributions you’ve likely never heard of it. Many times simpler is better and this was certainly not a simple format. Anyway, the scene had no interest, either.

Other Formats

Digging around it’s not hard to find other description formats that ultimately failed and/or had very short life spans:

DESC.SDN

This file can be found mentioned in various archive processing utilities such as RAScan 2.00 containing the following description (in it’s FILE_ID.DIZ no less):

RAScan v2.00    An online/offline upload
processor for RA 2.00 and higher. Support
for importing FILE_ID.DIZ, DESC.SDI,
DESC.SDN files into the RA Files Data Base.
Support for up to 10 compression programs.
Support for your choice of scanning program.
Support for commenting of archives. Much
more and more to come. Registration $10 U.S.

Another example is found within THD ProScan Version 10.0. Interestingly THD ProScan contains a DESC.SDN itself that appears to be a copy of the DESC.SDI file in the same archive.

.SDA Files

A small number of tools such as ExeMaster 3.8 mention inspecting .SDA files. This can be found referenced as “Fidonet Software Distribution Network Compressed Archive” and similar. Not much else is known.

Upload Processors

Another interesting (and controversial) thing that emerged was for BBS upload processing tools to not only process archives for possible viruses and description file extraction, but to add their own taglines to the description file (and add additional files to the archive as well). This was often used as a way to append advertisements or “We had it first” brags.

You can’t get much more meta than this: A FILE_ID.DIZ talking of adding 10 line FILE_ID.DIZ support with a “passed through” line appended by another system!

Some archive formats such as ZIP also had the ability to embed descriptions directly in the archive itself (vs a separate file within the archive). Many boards would use this mechanism to “tag” the file as well. Of interesting note however, this never took off as a replacement to the popular FILE_ID.DIZ. Or in other words, BBSes would write to the archive comment but not read from it. Users however, would generally see this when using PKUNZIP.

Apparently this file passed through The Stadium!

Closing

That about wraps up this “Retro Standards” entry. Hopefully you found it as interesting (and nerdy) as I did in researching and writing it. There are probably a lot of missing pieces of history here. If you have information that should be here please feel free to shout out!

Finally, a quick thanks to everyone on fsxNet, BBS groups on Facebook, etc. that helped me piece together some of this information – It was extremely helpful!