In the last three issues of TEXT Technology, "Column One" has contained descriptions of computer programs that were created using SPITBOL-386. This compiler for the SPITBOL computer language deserves to be described itself. Using SPITBOL-386, humanists can create simple (but genuinely useful) programs in a matter of minutes, and they can create complex software that reaches the limits of what can be done in non-numeric computing.
Two Program Examples
It has become increasingly common for publishers of electronic texts to embed tags
within the text in order to provide information about the work. For example, an electronic version of a play may contain tags to identify the lines assigned to each character and to mark the start and end of acts and scenes. Following a discussion of the problems of embedding tags in the electronic text of a play, I wondered just how many tags were used in typical texts. A play like Hamlet must require hundreds of tags, and the complete works of Shakespeare probably demand thousands. In a matter of minutes, I created a SPITBOL program that would tell me what I wanted to know. When the program was executed using the Electronic Edition of William Shakespeare: The Complete Works, I found (in about three seconds) that this edition of Hamlet had 1,934 tags. My program crunched away at the seven megabytes of the Complete Works for about one and one- half minutes, and it told me that the Works has 59,196 tags.
I could not have written a program that would do what I wanted to do with so little trouble in any computer language other than SPITBOL, and it executed really quickly. Figure 1 shows the entire program: nine lines. I constructed the program like a true humanist: I used the first code that came to mind, and I took few pains to make it efficient.
The logic of the program is rather elementary: it merely counts the number of opening angle brackets for tags. It asks the user for the name of a file, then it reads the file, line by line. Each line is searched, and if an angle bracket is found, it is deleted from the line (so that it is not counted a second time), and one is added to a counter. When there are no further lines to read, a message is printed telling the number of tags in the file. See Figure 1.
The simplicity of my tag-counter program is in contrast to another SPITBOL program I call CONCORD; it generates a complete key-word-in-context (KWIC) concordance for a text. Click here to read note 1. Figure 2 shows an extremely small part of CONCORD's output for Herman Melville's Moby-Dick. In order to produce a KWIC concordance for that novel, CONCORD identified each of the 210,349 words of running text, and it stored the words in a table along with the location of each word; it sorted them in alphabetical order, and it created an output file. CONCORD's output file containing a full KWIC concordance of Moby-Dick is over eighteen megabytes. As can be seen in Figure 2, CONCORD's output has each word centered in its context (of about thirty characters on each side), and the line number and word position in the line are given. It shows, for example, that the first occurrence of "white" in Moby-Dick is the tenth word in line 239 and it modifies "painting." Thus, CONCORD can provide detailed information for the researcher who wants to know where and how words in a text are used.
In order to process novel-length texts, CONCORD had to be written for a compiler that could access large amounts of memory and that would take advantage of the power of 32-bit processing that is available with 80386, 80486, and Pentium processors; and I wanted to use DOS or Windows (I certainly did not want to buy and learn a new operating system). SPITBOL-386 was the obvious choice. It runs under DOS and Windows; it can use all of the memory installed in a microcomputer (up to four gigabytes); and it has access to hardware memory paging and to 32-bit CPU registers. If SPITBOL-386 did not exist, I simply could not have created CONCORD.
Anyone who wants to do non-numeric computing on a DOS- or Windows-based 32-bit microcomputer should consider using SPITBOL-386, produced by Catspaw, Inc. Those who are familiar with SPITBOL programming will see the advantages of this new implementation immediately. SPITBOL-386 gives those who do not know it sufficient reason to learn it.
The SPITBOL-386 Compiler
SPITBOL is a speedy implementation of SNOBOL4 -- a programming language known for its unequaled pattern-matching and other functions that examine and manipulate strings of characters. The syntax and features of SPITBOL are almost exactly the same as those of SNOBOL4; the most significant difference is performance: SPITBOL executes programs six to ten times faster.
SPITBOL-386 is an extremely high-performance implementation of SPITBOL. SPITBOL has been available for MS-DOS computers since 1984, but its chief drawback is that its memory constraints severely limit the size of programs and their data. SPITBOL-386 has no such constraints; it not only allows the use of the full 640 KB of conventional memory, but by incorporating DOS extenders, SPITBOL-386 also allows the use of all extended memory up to four gigabytes. I have run programs under SPITBOL-386 that use every byte of the sixteen megabytes installed in my computer.
The maximum number of bytes of memory used by any one string in a SPITBOL program is restricted to the value of the keyword &MAXLNGTH.; SPITBOL-386's default value of 8,388,608 for &MAXLNGTH; indicates that this is a compiler that may be expected to process enormous amounts of data. (Moreover, &MAXLNGTH; may be set to a value higher than the default, should it be necessary.)
In part because it uses all available memory, SPITBOL-386 executes programs even more rapidly than other versions of SPITBOL. SPITBOL-386 is the fastest SPITBOL compiler available for a microcomputer, and it may very well be the quickest compiler for non-numeric computing available for a microcomputer. Only about five minutes is needed for CONCORD to produce an 11. Click here to read note 2. megabyte KWIC concordance for Lord Jim when it is run on a (now, rather modest) microcomputer containing an 80486-25 MHZ processor. If it were printed, the 11.2 megabyte concordance would require about 2,500 sheets of paper: five reams. Thus, this SPITBOL-386 program can produce about five hundred pages of output per minute (no microcomputer printer could keep up with such output, of course). It should execute two or three times faster on a Pentium machine.
Programs executed under SPITBOL-386 run about ten times faster than almost identical programs under SNOBOL4. At least for non-numeric applications, SPITBOL-386 execution times are comparable to those of C programs -- and SPITBOL programs are much easier to write and understand than those for C (at least for this programmer).
SPITBOL-386 has forty-one special functions (called "HOST functions") to perform specialized operations on the host computer -- such as executing a DOS command, writing directly to the screen buffer, and immediately accessing memory locations. Thirty of these functions are contained in earlier implementations of SPITBOL.
One of the eleven HOST functions added by SPITBOL-386 allows issuing DOS commands from within a SPITBOL program. Thus the execution of a SPITBOL program can be interrupted -- and files can be deleted or copied, a snapshot of current memory use can be taken, or another program can be run -- and then the execution of the SPITBOL program can be resumed. Other added HOST functions are extended screen calls that get information about the video adapter and display and that determine the relation between the screen buffer and the physical screen. Using these functions, the screen buffer can be used to establish a virtual screen that has a greater width than the width of the physical screen; the virtual screen can then be scrolled horizontally across the physical screen.
Much like the C language, SPITBOL-386 implements an INCLUDE control statement that allows programmers to specify text files to be included as a part of the compiled program. Some files designed to be included in SPITBOL programs are contained on the distribution disk. If the file named HOST.INC is included with a SPITBOL-386 program, it allows the substitution of functions with mnemonic names for the HOST functions. For example, the EXECUTE function can be used to execute a DOS command (from within a SPITBOL-386 program) in place of the HOST(1) function.
Although users will probably select SPITBOL-386 for its non- numeric processing abilities, its numeric computing power is not unimpressive. In all calculations, it maintains accuracy to fifteen decimal places for real numbers which may be as large as 0.18E+309 or as small as 0.22E-309. It has built-in trigonometric and logarithmic functions, and it allows the same numeric comparisons and operations as most other languages.
SPITBOL-386 and DOS Extenders
Currently, there is some confusion about the best ways to make use of the great power and huge memory available in computers based on 32-bit (80386, 80486, and Pentium) microprocessors. There are operating systems (such as Unix -- which has its own version of SPITBOL) that can be used to control the power and memory of the new processors. However, programmers in the humanities who use DOS and Windows, tend to view the steep learning curve of a new operating system with dismay.
SPITBOL-386 incorporates DOS extenders. When SPITBOL-386 is run, the extender switches the 32-bit processor into protected mode in which programs have access to all available memory (up to four gigabytes); they can use the 32-bit 80386 instruction set, and they can access hardware memory paging. When SPITBOL-386 must use DOS (such as for file input or output), the extender switches the processor out of protected mode, DOS moves data (or does whatever is necessary), and the protected mode is resumed. All of this is invisible to SPITBOL programmers. SPITBOL-386 is run like any DOS compiler, but its programs can use all available memory, and they can take full advantage of the processing power and speed of 80386, 80486, and Pentium processors.
The purchaser of SPITBOL-386 actually receives two compilers: I will call them SPITBOL-P and SPITBOL-I. In most ways, the two compilers are identical, and both run fine under native DOS (with no multitasking or memory management systems), but there are important differences. SPITBOL-P contains a DOS extender from PharLap Software, and it is compatible with a memory manager such as QEMM-386 and 386MAX. SPITBOL-I contains a DOS extender from Intel Corporation, and it is compatible with Windows.2 I have found that one or the other or both versions of SPITBOL-386 can be made to work in most commonly-used environments, including OS/2. Click here to read note 3.
SPITBOL-I, like older versions of SPITBOL, is capable of creating stand-alone, executable files that can be distributed royalty-free by programmers. It is possible to save files created by SPITBOL-P after compilation, but in order to be used, these save files must be run with the full SPITBOL-386 compiler or with a special runtime program that must be purchased.
By swapping pages of memory to and from disk as necessary, both SPITBOL-P and SPITBOL-I can execute programs that require more memory than is provided in a microcomputer's RAM. That is, for example, a program that holds ten megabytes in memory as it executes can be run on a machine with only eight megabytes of RAM. This sounds like a terrific feature (and it is), but there are two drawbacks. First, users of SPITBOL-P must purchase PharLap's 386|VMM in order to use such virtual memory. Second, and more serious, the speed of execution of a program that requires swapping to disk can be so slow that it is simply unacceptable (the manual contains a warning about this). I attempted to execute a version of COLLATE with data that would require it to hold a sixteen-megabyte table in memory on a computer with only eight megabytes of RAM. It processed about half of the data in about fifteen minutes, but then as it began swapping to disk, it slowed to a rate that would have required days (or even weeks) to complete its execution.
Neither version of SPITBOL-386 includes a debugger or editor. Nevertheless, debugging can be accomplished fairly easily from within a program by using SPITBOL's DUMP, PROFILE, and TRACE functions, and there are many good programming editors marketed, and most word processors can be used to produce program files.
Documentation and Support
The indexed 336-page printed manual is clear, well- organized, and helpful. Aside from a brief introduction ("Getting Started") and appendices, it is divided into two parts: a tutorial and a reference manual. The tutorial is a no- nonsense, straight-forward guide to programming in SPITBOL. It does not claim to be sufficient for a user with absolutely no computer background, but a diligent student who has even a vague understanding of programming can probably use it to create useful SPITBOL programs. Click here to read note 4. The reference section is an unabridged description of SPITBOL-386.
There is a (perhaps unavoidable) drawback of the manual. It covers all implementations by Catspaw of SPITBOL for 32-bit computers; thus SPITBOL-386 users must sometimes read around instructions that are intended only for Unix implementations or for RISC systems.
Catspaw, Inc., is known for its excellent technical support. The phone number, mailing address, and electronic mail address are listed prominently in the manual. Colleagues and I have found that assistance by phone can almost always be obtained at any time during business hours; answers to questions sent by regular mail or by electronic mail are usually sent out the same day they are received.
Conclusion
Because of its ability to process huge amounts of non- numeric data quickly, SPITBOL-386 may be the language of choice for many. In addition to its memory capacity and speed, this modern implementation of SNOBOL4 includes all of the features and extensions of earlier SPITBOL versions: it has automatic data type conversion, management and regeneration of memory, and powerful built-in string functions. Catspaw, Inc., is respected for its good documentation and technical support. Humanists may have their reasons for not using SPITBOL-386, but it is a compiler that demonstrates the performance, automatic functions, and support that should be demanded of any programming language used in the 1990's and after.
Name: SPITBOL-386: The High-Performance SNOBOL4 Language. Version Reviewed: Release 3.7 (ver 1.27) System Requirements: Microcomputer with 80386, 80486, or Pentium processor; MS-DOS version 3.1 or above; SPITBOL-386 will run in 640 KB of conventional RAM, but some extended memory is recommended; it will use all available extended memory up to four gigabytes. Manual: 336 pages (7" x 9 1/2") in three-ring binder, indexed. Price: $295.00. Company: Catspaw, Inc. P.O. Box 1123 Salida, CO 81201 U.S.A. (719) 539-3884 fax: (719) 539-4830
1. I am very grateful to Mark Emmer of Catspaw, Inc., for his assistance with CONCORD.
2. SPITBOL-P is VCPI-compliant; that is, it is designed to work with software that complies with the Virtual Control Program Interface protected-mode memory standard. SPITBOL-I is DPMI- compliant; it is designed to work with software that supports the DOS Protected Mode Interface standard.
3. Catspaw, Inc., also produces an OS/2 version of SPITBOL.
4. For an introduction to computer programming using SNOBOL4 or SPITBOL see Susan Hockey, SNOBOL Programming for the Humanities (New York, Oxford University Press, 1985).
Eric Johnson is the author or editor of over one hundred volumes and articles, and he is the creator of commercial software written in SPITBOL. He can be contacted via email as JohnsonE@jupiter.dsu.edu.
Click here to go to Eric Johnson's publications.
Click here to go to Eric Johnson's home page.
MAKE.TPAT TPAT = BREAK("<") LEN(1)
GET.NAME TERMINAL = "Enter the name of the file:"
FILE.NAME = TERMINAL
INPUT(.IN,1,FILE.NAME) :F(GET.NAME)
READ LINE = TRIM(IN) :F(OUT)
FIND.TAG LINE TPAT = :F(READ)
COUNT = COUNT + 1 :(FIND.TAG)
OUT TERMINAL = "There are " COUNT " tags in " FILE.NAME
END
Figure 1. A SPITBOL program to count the number of tags in a text.
239.10 nging sign over the door with a WHITE painting upon it, faintly repre
418.6 ply brown and burnt, making his WHITE teeth dazzling by the contrast;
609.4 me. I remembered a story of a WHITE man --a whaleman too--who, fall
616.12 heard of a hot sun's tanning a WHITE man into a purplish yellow one.
1297.11 e yet afloat. And ever, as the WHITE moon shows her affrighted face
1330.12 observe his prayer, and so many WHITE bolts, upon his prison. Then J
1733.10 e so companionable; as though a WHITE man were anything more dignifie
1864.14 starboard hand till we opened a WHITE church to the larboard, and the
3003.2 in the moonlight; and like the WHITE ivory tusks of some huge elepha
3046.9 eedlessly, ye harpooneers; good WHITE cedar plank is raised full thre
3462.8 ity in looking up at him; and a WHITE man standing before him seemed
3462.15 an standing before him seemed a WHITE flag come to beg truce of a for
3556.2 ers of discernment. So that no WHITE sailor seriously contradicted h
3562.5 mness was owing to the barbaric WHITE leg upon which he partly stood.
.
.
.
8274.6 -head. In the distance, a great WHITE mass lazily rose, and rising hi
8281.10 he breaches! right ahead! The WHITE Whale, the White Whale! Upon t
8282.2 ht ahead! The White Whale, the WHITE Whale! Upon this, the seamen r
8292.3 did he distinctly perceive the WHITE mass, than with a quick intensi
8307.13 m, than to have seen thee, thou WHITE ghost!
.
.
.
16362.12 c fountain in his head, did the WHITE Whale now reveal his vicinity;
16370.3 his immeasureable bravadoes the WHITE Whale tossed himself salmon-lik
16390.14 p's three masts to his eye; the WHITE Whale churning himself into fur
16399.3 his untraceable evolutions, the WHITE Whale so crossed and recrossed,
16414.12 fast again. That instant, the WHITE Whale made a sudden rush among
16429.6 rpendicularly from the sea, the WHITE Whale dashed his broad forehead
Figure 2. Output from CONCORD: a KWIC concordance program written in SPITBOL.
