Inside a SAVEd Atari BASIC Program: Tokens, Tables and LIST Protection
When you press RETURN on a line of Atari BASIC, the cartridge does not keep what you typed. It tokenizes the line: keywords become single bytes, variable names become one-byte references into a table, and numbers become six-byte floating-point constants. LIST rebuilds readable text from those bytes each time. Knowing the format lets you write utilities that inspect or patch programs, explains several odd behaviours, and shows how "unlistable" programs were made and how to rescue them.
What you'll learn: the memory pointers that describe a program, the layout of a tokenized line, the main token values, the variable name and value tables, the SAVE file format, and the two classic LIST-protection tricks, with a tested repair program.
What you need: Atari BASIC (every example here was run on revision C) and a disk or the emulator's H: device for the file examples.
1. The pointers in page zero
BASIC keeps a set of two-byte pointers in zero page. Each is read as PEEK(n)+256*PEEK(n+1):
| Address | Name | Points to |
|---|---|---|
| 128–129 | LOMEM | BASIC's low memory: a 256-byte token buffer, then the program |
| 130–131 | VNTP | Start of the variable name table |
| 132–133 | VNTD | End of the name table (the dummy zero byte that ends it) |
| 134–135 | VVTP | Start of the variable value table |
| 136–137 | STMTAB | Start of the program (the statement table) |
| 138–139 | STMCUR | The line currently being executed or entered |
| 140–141 | STARP | Start of the string and array area |
| 142–143 | RUNSTK | The GOSUB/FOR stack |
| 144–145 | MEMTOP | Top of BASIC's used memory |
The regions follow each other in that order: names, values, program, then the strings and arrays you DIM at run time.
2. How a line is stored
Every line starts with three bytes: the line number (low byte, high byte) and the total length of the line in bytes. You can therefore walk a program by adding each length to the line's address. Line number 32768, one more than you can type, marks the direct-mode line that always sits at the end. Larry Isaacs used exactly this walk in "Inside Atari BASIC" (COMPUTE!'s First Book of Atari).
Each statement then starts with a statement offset byte: the distance from the start of the line to the start of the next statement. A line with one statement has the same value in its third and fourth bytes. After the offset comes the statement token, its operands, and a terminator: 20 for a colon or 22 for the end of the line. REM and DATA are different. Their text is stored as typed and ends with the ATASCII end-of-line character, 155.
This program walks its own lines and prints the bytes, then lists the variable names:
10 X=25:PRINT "HI"
20 DIM N$(10),A(3)
100 S=PEEK(136)+256*PEEK(137)
110 L=PEEK(S)+256*PEEK(S+1):IF L>=100 THEN 150
120 ? "LINE ";L;":";:FOR I=2 TO PEEK(S+2)-1:? " ";PEEK(S+I);:NEXT I
130 ? :S=S+PEEK(S+2):GOTO 110
150 ? "NAMES:";:V=PEEK(130)+256*PEEK(131)
160 C=PEEK(V):IF C=0 THEN ? :END
170 IF C>127 THEN ? CHR$(C-128);" ";:V=V+1:GOTO 160
180 ? CHR$(C);:V=V+1:GOTO 160
Line 10 comes out as 22 15 54 128 45 14 64 37 0 0 0 0 20 22 32 15 2 72 73 22. Decoded:
| Bytes | Meaning |
|---|---|
| 22 | Line length (the two line-number bytes 10, 0 are skipped by the program) |
| 15 | First statement ends at offset 15 |
| 54 | The invisible LET of X=... |
| 128 | Variable 0, which is X |
| 45 | = used as numeric assignment |
| 14, then 64 37 0 0 0 0 | A numeric constant follows: 25 |
| 20 | Colon: end of statement |
| 22 | Second statement ends at offset 22, the end of the line |
| 32 | |
| 15, 2, 72 73 | A string constant: length 2, then "HI" |
| 22 | End of line |
3. The tokens
Statement tokens (the byte after each statement offset) run from 0 to 55, in this order:
0 REM, 1 DATA, 2 INPUT, 3 COLOR, 4 LIST, 5 ENTER, 6 LET, 7 IF, 8 FOR, 9 NEXT, 10 GOTO, 11 GO TO, 12 GOSUB, 13 TRAP, 14 BYE, 15 CONT, 16 COM, 17 CLOSE, 18 CLR, 19 DEG, 20 DIM, 21 END, 22 NEW, 23 OPEN, 24 LOAD, 25 SAVE, 26 STATUS, 27 NOTE, 28 POINT, 29 XIO, 30 ON, 31 POKE, 32 PRINT, 33 RAD, 34 READ, 35 RESTORE, 36 RETURN, 37 RUN, 38 STOP, 39 POP, 40 ?, 41 GET, 42 PUT, 43 GRAPHICS, 44 PLOT, 45 POSITION, 46 DOS, 47 DRAWTO, 48 SETCOLOR, 49 LOCATE, 50 SOUND, 51 LPRINT, 52 CSAVE, 53 CLOAD, 54 implied LET, 55 ERROR- (a line BASIC could not parse).
Operand tokens run from 14 to 84. The ones you meet most:
| Token | Meaning | Token | Meaning |
|---|---|---|---|
| 14 | Numeric constant (6 bytes follow) | 27 | THEN |
| 15 | String constant (length byte, then text) | 28 | # |
| 18 | Comma | 29–34 | <= <> >= < > = (numeric comparison) |
| 20 | End of statement (colon) | 35–39 | ^ * + - / |
| 21 | Semicolon | 40–42 | NOT, OR, AND |
| 22 | End of line | 43, 44 | ( and ) |
| 23, 24 | GOTO, GOSUB inside ON | 45, 46 | = assigning a number, a string |
| 25, 26 | TO, STEP | 47–52 | the same comparisons for strings |
Tokens 61 to 84 are the functions, from STR$ (61), CHR$, USR, ASC, VAL, LEN, ADR, ATN, COS, PEEK, SIN, RND, FRE, EXP, LOG, CLOG, SQR, SGN, ABS, INT, PADDLE, STICK, PTRIG to STRIG (84). Several symbols have more than one token depending on use. = alone has four: assignment of a number (45), assignment of a string (46), numeric comparison (34) and string comparison (52). That is how BASIC knows at run time what each one means without re-parsing.
Any byte of 128 or more in an expression is a variable: 128 is the first name in the table, 129 the second, and so on up to 255. That is where the limit of 128 variable names comes from.
Numbers are stored in the OS's six-byte BCD floating-point format. The first byte is an exponent in powers of 100, offset by 64, with bit 7 as the sign. The next five bytes hold ten decimal digits, two per byte. In the dump, 25 is 64 37 0 0 0 0: exponent 64 means 100⁰, and 37 is hexadecimal $25.
4. The variable name and value tables
The name table holds each name in the order it was first typed. The last character of each name has bit 7 set. String names end in $ and array names in (, both stored with bit 7 set. A zero byte ends the table, and VNTD points at that zero. Names stay in the table after you delete every line that used them. Only LIST to a file, NEW and ENTER rebuilds it (see Saving, Loading and Merging Atari BASIC Programs).
The value table has eight bytes per variable, in the same order:
| Byte 1 | Byte 2 | Bytes 3–8 |
|---|---|---|
| 0 = number | variable number | the six-byte value |
| 64 = array (65 once DIMmed) | variable number | offset from STARP, first DIM+1, second DIM+1 |
| 128 = string (129 once DIMmed) | variable number | offset from STARP, current length, DIM size |
Because a variable's eight bytes are found by arithmetic on its number, BASIC never has to search for a value at run time.
5. The SAVE file format
A SAVEd file is a direct copy of this memory with a short header. The header is seven two-byte pointers, LOMEM through STARP, with LOMEM subtracted from each, so the file always starts with two zero bytes. The name table, value table, program and direct-mode line follow. String and array contents are not saved. On LOAD, BASIC adds the current value of MEMLO back to each pointer. That is why a program SAVEd under one DOS loads correctly under another that uses more or less low memory.
The test program above, SAVEd to disk, began 00 00 00 01 06 01 07 01 37 01 .... LOMEM is 0, the name table starts at $0100 (the 256-byte token buffer comes first), and so on. Byte 14 of the file is the first byte of the name table.
6. How LIST protection works
Two tricks were common in the 1980s. Both rely on BASIC running happily from tokens while LIST needs everything to be consistent.
Scrambling the name table. Running tokens refer to variables by number, so the names themselves are never needed at run time. Overwrite them and the program still runs, but LIST prints rubbish where the names were. Carl Evans's Atari BASIC Faster and Better gives a short routine of this kind, while asking readers not to use it.
Zeroing a line length. Atari Software Protection Techniques recommends adding this as the last line and running it with GOTO:
32500 POKE PEEK(138)+256*PEEK(139)+2,0:SAVE "D:PROGRAM":NEW
STMCUR points at the line being executed, so this sets that line's length byte to zero before saving. Here is what happened when we tried it, combined with a name-scrambling loop, on a test program:
RUN "D:PROGRAM"loads and runs the program normally, because execution simply moves from statement to statement.LOAD "D:PROGRAM"followed by any direct command, including LIST and RUN, locks the machine. To store the command, BASIC walks the line chain to find where line 32768 belongs. A length of zero means it never moves past the damaged line.
That is why protected programs told you to type RUN "D:..." or RUN "C:" and nothing else.
7. Repairing a protected file
The fix is to work on the file rather than on memory. This program reads a SAVEd file into a string, recomputes each line's length from its statement offsets, invents a fresh name for every variable (N0, S1$, A2( and so on, according to the type byte in the value table), and writes a corrected file. Because the new name table may be a different size, it shifts the later pointers by the difference.
10 REM UNPROTECT A SAVED BASIC FILE
20 DIM F$(20),B$(6000),N$(12),T$(3),W(6)
30 ? "SOURCE FILE";:INPUT F$
40 OPEN #1,4,0,F$:TRAP 60:N=0
50 GET #1,X:N=N+1:B$(N)=CHR$(X):GOTO 50
60 TRAP 40000:CLOSE #1:? N;" BYTES READ"
70 FOR K=0 TO 6:W(K)=ASC(B$(2*K+1))+256*ASC(B$(2*K+2)):NEXT K
80 B=W(1)-15:REM ADDRESS A IS AT B$(A-B)
90 NV=(W(4)-W(3))/8:? NV;" VARIABLES"
100 REM REPAIR LINE LENGTHS
110 P=W(4)-B:E=W(6)-B
120 IF P>=E THEN 200
130 Q=P+3
140 S=ASC(B$(Q)):C=ASC(B$(P+S-1))
150 IF C<>22 AND C<>155 THEN Q=P+S:GOTO 140
160 IF ASC(B$(P+2))<>S THEN ? "LINE ";ASC(B$(P))+256*ASC(B$(P+1));" REPAIRED":B$(P+2,P+2)=CHR$(S)
170 P=P+S:GOTO 120
200 REM BUILD NEW VARIABLE NAMES
210 DIM V$(NV*6+1):V$=""
220 FOR I=0 TO NV-1
230 T=ASC(B$(W(3)-B+I*8)):T$=STR$(I)
240 IF T<64 THEN N$="N":N$(2)=T$
250 IF T>=64 AND T<128 THEN N$="A":N$(2)=T$:N$(LEN(N$)+1)="("
260 IF T>=128 THEN N$="S":N$(2)=T$:N$(LEN(N$)+1)="$"
270 N$(LEN(N$))=CHR$(ASC(N$(LEN(N$)))+128)
280 V$(LEN(V$)+1)=N$:NEXT I
290 D=LEN(V$)+1-(W(3)-W(1))
300 ? "OUTPUT FILE";:INPUT F$
310 OPEN #2,8,0,F$
320 FOR K=0 TO 6:A=W(K):IF K=2 THEN A=W(1)+LEN(V$)
330 IF K>2 THEN A=A+D
340 PUT #2,A-256*INT(A/256):PUT #2,INT(A/256):NEXT K
350 FOR I=1 TO LEN(V$):PUT #2,ASC(V$(I)):NEXT I:PUT #2,0
360 FOR I=W(3)-B TO N:PUT #2,ASC(B$(I)):NEXT I
370 CLOSE #2:? "DONE"
Run it, then give it the names of the protected file and a new file, such as D:GAME.BAS and D:GAME2.BAS. On our test file, which had both its names and its last line length destroyed, the repaired copy LOADed and LISTed in full, with names such as S0$, A1( and N2. You then rename the variables by hand as you work out what each one does.
Line 140 is the key idea. Each statement offset points to the next statement, so following the offsets until one lands just after a 22 or 155 gives the true line length, whatever the length byte says.
Troubleshooting
- ERROR 9 in the repair program: the file is bigger than B$. Raise the DIM in line 20 (memory permitting), or switch to the block-read method in CIO and IOCBs for big files.
- The repaired program still won't LIST: the protector may have damaged something else, such as a line number or a statement token. Compare the file with the layout in section 2 using a hex viewer on a PC.
- "NOT ATARI BASIC" or strange tokens: Turbo-BASIC XL and other extended BASICs use extra token values, so their SAVEd files differ from Atari BASIC's.
- Line walk never ends in your own utilities: stop at line number 32768 or at STMCUR, and treat a zero length as damage.
Further reading
- Larry Isaacs, "Inside Atari BASIC", and W. A. Bell, "Atari BASIC Structure", both in COMPUTE!'s First Book of Atari (1981).
- The Atari BASIC Source Book by Bill Wilkinson, Kathleen O'Brien and Paul Laughton (COMPUTE! Books, 1983): the cartridge's own annotated source.
- Ian Chadwick, Mapping the Atari: locations 128–145 in detail.
- Carl M. Evans, Atari BASIC Faster and Better (IJG, 1983): variable-table utilities and the scramble routine.
- On this site: Saving, Loading and Merging Atari BASIC Programs, Making Atari BASIC Faster (and Smaller), Self-Modifying Atari BASIC: Forced-Read Mode and Programs That Write Programs.
Credits
- Picture: The example program walks its own statement table and prints each line's bytes, then the variable name table. — Program authors; screenshot by the site
- Larry Isaacs, "Inside Atari BASIC", COMPUTE!'s First Book of Atari (1981) — Larry Isaacs; AtariArchives.org, 1981
- Mapping the Atari (Ian Chadwick) — locations 128-145 (LOMEM … MEMTOP), SAVE/LOAD pointer handling, VVT layout — AtariArchives.org
- Site tool detok.py (token tables verified against the emulator's LIST output)
