This is the final part of a three-part series.

In this post I’m going to discuss Fire Escape’s BBS directory formats, the directory parser and the dataset; I’ll also give some caveats about this data.

You can read the previous installments here:

• Part 1: Fire Escape’s BBS Directory (introduction)
• Part 2: The St. Louis scene in charts

The directories

Fire Escape’s directories are plain text files, with information laid out in columns. Over the ten years of the directories, she employed three different layouts/formats. I will describe these in a moment.

The directories were distributed in ZIP archives. Each ZIP might contain these files:

• BBSXXX93.TXT – That month’s directory (XXX = month)
• UPDXXX93.TXT – A condensed BBS update. These documents are very interesting, because they list reasons Fire Escape removed BBSes from that month’s directory, as well as delineate various other changes. Each update also contained a “BBS Tip of the Month,” like “Do NOT send $$$ to a BBS that has been up LESS than 6 months!” Unfortunately, I have only been able to find four of these Update files. I assume there was one every month.
• FE-VIEW.EXE – This was a special ANSI viewer for the BBS directories, written by Paul Hartman. Fire Escape stopped including it in her directory ZIP files around March 1994 because it was causing “false positive virus alerts.”

First format

It’s unclear how many editions of Fire Escape’s BBS Directory used this first format, because only one edition survives from prior to Nov. 1992. This initial format listed a BBS’s phone number, name, sysop, baud rate, and software type.

Second format

The second format recorded more information: BBS name, number of lines, toll info, phone number, sysop name, hours, modem type, and software type. Best of all, there was a “Stats” column where Fire Escape included codes to indicate whether a BBS required paid memberships, offered RIP graphics support, had internet access, and other things. The notes she recorded in this column varied over the years. Also, Fire Escape would append the word “ADULT” in all caps to the names of BBSes that offered “sexually-explicit files and topics.” In contrast, the word “Adult” in title case indicated only that a board was closed to children. This format began in Nov. 1992 (or earlier) and was in use through Dec. 1996.

Third format

This final format was somewhat simpler. It eliminated the separate column for toll info, to allow for an expanded “Codes” column. “Codes” was similar to “Stats” from the 2nd format, but offered many more possible codes to indicate if a board was religious, adult, offered telechat, had lots of games, etc. The third format was in place from Jan. 1997 until the Jan. 2000 final edition.

The third format offered the most information about individual BBSes. Unfortunately, it debuted several years after the height of the scene.

The parser

Don’t call me a hacker; a hack might be more like it.

I know my way around Javascript and to a lesser extent PHP, and so that is what I used to write my parser. I know, I know. In the future I plan to re-implement the parser in Python as a project to learn that language. But for now, this has done the trick.

Basically, PHP grabs the text content of the directories and adds them to an HTML document as <pre> elements. I then use Javascript and regular expressions to parse the files line by line. The parser is sensitive to the three formats, and also knows about the variations in the “stats”/”codes” Fire Escape used over the years.

The parser consolidates some of the data. For example, BBSes with multiple telephone numbers were given a separate line of text in the directory for each phone number. Simply counting the lines in a directory to get a count of BBSes would result in an inflated total. So, the parser consolidates all lines of text for a BBS into just one record for that month, with all of its phone numbers recorded in the phone field.

The parser also translates/normalizes data. There is variation among the BBS software codes Fire Escape used over the years, so I have consolidated some codes. For example, “ROBO” and “RBFX” are consolidated into “RoboBoard” in my database. Similarly, I have tried to translate the various modem speed codes into one “top speed.” For example, I convert “14.4,” “v.14,” “HST,” “32b,” “DSH,” “DSC,” and “DS32” each into “14400.”

There’s still a lot the parser isn’t doing yet. Among the items on my to-do list are: parsing information in parentheses or brackets in a BBS’s name field; writing logic to determine how many total lines a multiline BBS had; and parsing the “hours” fields.

The data

The final combined dataset has more than 17,000 rows. Each row represents one BBS in St. Louis during one month.

I uploaded the dataset to MySQL and used SQL queries to build summary series that I fed into Highcharts.js to create the St. Louis BBS scene in charts blog post.

I am not an expert data analyst, but I’ve done my best to parse the data faithfully. Undoubtedly there are mistakes and areas where the dataset could be improved.

In the spirit of “open data,” I’m making the dataset available to download. I welcome any corrections, criticisms, or suggestions folks might have. Hopefully it might also be of use or interest to researchers who might want to go further or approach it differently.

The downloads:

• Fire Escape’s BBS directories 1991-2000 [ZIP]
• The combined dataset [CSV]
• The combined dataset [JSON]

In the future when I improve the parser, I will update these links so you can download the newest revisions of the dataset.

The caveats

As thorough as Fire Escape was in compiling this list, there were certainly BBSes that she missed or that she excluded, for various reasons.

For example, I compared the May 1995 edition of Fire Escape’s BBS directory with the May 1995 edition of the USBBS list USBBS was a national listing of “PC Bulletin Boards for DOS Users.” I found that USBBS listed seven 314-area BBSes that were never listed in one of Fire Escape’s directories. I found another seven BBSes which weren’t listed in Fire Escape’s May 1995 directory, but had been listed in previous editions of the directory.

Similarly, as I have tried to describe above, a lot of the metadata that Fire Escape recorded changed over the years.

For example, I am finding the “Adult” category to be among the trickiest. During the years of the second format (~1992 through 1996), the “ADULT” label seems anomalous in that it declines from 11 BBSes in Dec. 1992 to 1 in much of 1995. With the advent of the third format 1997, the number suddenly jumps back up to 11. It seems unlikely to me that the number of adult boards declined; instead, I suspect Fire Escape wasn’t labeling them consistently until she added a dedicated code for it in the third format. Of course, I could be wrong.

Anyway, to sum up: There’s no doubt that the data is a little bit incomplete, and in places a bit inconsistent.

But Fire Escape’s data is thorough and gives us a great picture of the St. Louis BBS scene — likely the best picture we’ll ever get.