ANTIC Modes 4 and 5: Multicolour Character Graphics in Depth

An ANTIC mode 4 screen: white clouds on a blue-green sky, small green trees, a band of grass, a red brick wall with white mortar, dark blue water and a white ice field

ANTIC modes 4 and 5 are the workhorses behind a huge number of Atari 8-bit games. They look like text modes to the hardware, 40 bytes per line with each byte naming a character, but every character is drawn as a little 4 × 8 block of coloured pixels instead of a one-colour letter. You get a 160-pixel-wide, five-colour screen for under 1K of screen memory, and you can redraw a whole room by changing a few hundred bytes.

Redefined Character Sets covers copying and editing fonts and briefly introduces the four-colour modes. This guide goes further: exactly how the bits map to colours, how to use the fifth colour, building mode 4 and mode 5 display lists by hand, drawing screens from tile maps, and the tricks of designing a 4-colour font.

What you'll learn: the bit-pair encoding and the inverse-character PF3 switch, a complete BASIC tile screen, the mode 5 variant, a MADS program that expands a metatile map, and design rules for multicolour fonts.

What you need: any Atari 8-bit or an emulator, Atari BASIC, and optionally the MADS assembler. Familiarity with Display Lists Explained helps.

1. The two modes at a glance

ANTIC 4 ANTIC 5
BASIC (XL/XE OS) GRAPHICS 12 GRAPHICS 13
Scan lines per row 8 16 (each font row shown twice)
Rows on a full screen 24 12
Bytes per row (normal width) 40 40
Character cell 4 × 8 pixels 4 × 16 pixels
Character set 128 characters, 1K, 1K-aligned same

Each pixel is one colour clock wide, twice the width of a GRAPHICS 0 dot. So a full mode 4 screen is 160 × 192 pixels, the same horizontal resolution as GRAPHICS 7 and 15. It needs 960 bytes of screen memory plus a 1K font, against 7,680 bytes for a GRAPHICS 15 bitmap. Mode 5 is identical except that ANTIC repeats each font row, so the characters are twice as tall and a full screen needs only 480 bytes.

On the XL/XE operating system these are GRAPHICS 12 and 13 (add 16 for no text window). The 400/800 OS doesn't know them, so there you build or patch the display list yourself, as shown below.

2. Bit pairs and the fifth colour

Each byte of character data describes one row of four pixels, two bits per pixel, leftmost pixel in bits 7–6:

Bit pair Screen byte bit 7 = 0 Screen byte bit 7 = 1 ("inverse")
00 COLBK (712) COLBK (712)
01 COLPF0 (708) COLPF0 (708)
10 COLPF1 (709) COLPF1 (709)
11 COLPF2 (710) COLPF3 (711)

So the row byte %00011011 is background, PF0, PF1, PF2 from left to right. The character name in screen memory supplies the extra bit: bit 7 does not invert the character as it does in GRAPHICS 0. Instead it switches the 11 pixels from PF2 to PF3. That is how mode 4 reaches five colours on a line. Any one character can show only four (background, PF0, PF1 and either PF2 or PF3), but neighbouring characters can differ.

Two consequences matter in practice:

  • Only bit pair 11 is "switchable". Put the colour you want to vary on 11. Orson Scott Card's classic example is an apple tree with fruit drawn in 11: print the character normally for red apples, inverse for orange ones, without defining a second character.
  • CHACTL (755) has no effect on blinking or inversion in modes 4–7. Its bit 2 (upside-down characters) still works in every character mode.

Collision detection follows the colours. A player touching 11 pixels of an inverse character registers a PF3 collision, not PF2, which gives you a free way to tell two otherwise identical tiles apart in hardware.

3. A mode 4 tile screen from BASIC

This program reserves 2K below RAMTOP (putting the font in the upper 1K, out of reach of the OS's screen clearing), opens GRAPHICS 0 and patches its display list to mode 4. Since mode 4 has the same 24 × 40 layout as GRAPHICS 0, nothing else has to move. It then defines five characters and pokes a scene straight into screen memory:

10 REM ANTIC 4 TILE SCREEN
20 A=PEEK(106)-8:POKE 106,A:GRAPHICS 0
30 CH=A+4:CS=CH*256:POKE 752,1
40 DL=PEEK(560)+256*PEEK(561)
50 POKE 559,0:POKE DL+3,68
60 FOR I=DL+6 TO DL+28:POKE I,4:NEXT I
70 FOR I=0 TO 7:POKE CS+I,0:NEXT I
80 FOR I=8 TO 47:READ D:POKE CS+I,D:NEXT I
90 POKE 756,CH:POKE 708,198:POKE 709,36
100 POKE 710,14:POKE 711,132:POKE 712,150
110 SC=PEEK(88)+256*PEEK(89)
120 FOR I=0 TO 959:POKE SC+I,0:NEXT I
130 FOR X=2 TO 38 STEP 6:POKE SC+560+X,4
140 POKE SC+X*3+120,5:POKE SC+X*3+121,5:NEXT X
150 FOR X=0 TO 39:POKE SC+600+X,2
160 POKE SC+640+X,1:POKE SC+680+X,1
170 FOR Y=18 TO 20:POKE SC+Y*40+X,131:NEXT Y
180 FOR Y=21 TO 23:POKE SC+Y*40+X,3:NEXT Y
190 NEXT X:POKE 559,34
200 GOTO 200
210 DATA 171,171,171,255,186,186,186,255
220 DATA 17,85,85,153,170,170,170,170
230 DATA 207,255,255,252,255,255,63,255
240 DATA 20,85,117,87,85,20,40,40
250 DATA 0,60,255,255,60,0,0,0

How it works:

  • Line 50 turns off screen DMA while drawing (POKE 559,0) and changes the LMS instruction at DL+3 from 66 (mode 2 + 64) to 68 (mode 4 + 64). Line 60 changes the remaining 23 mode bytes to 4.
  • Line 70 clears character 0, so the zeroed screen shows plain background. Line 80 reads characters 1–5 (brick, grass, water/ice, tree, cloud) into the RAM set. The rest of the set is whatever was in memory; that doesn't matter as long as those characters never appear on screen.
  • Lines 90–100 point CHBAS at the set and set the five colours: green PF0, brown PF1, white PF2, blue PF3 and a blue-green background.
  • Lines 170–180 fill rows 18–20 with character 131 (3 + 128) and rows 21–23 with plain 3. The data are identical, but the inverse rows show their 11 pixels in PF3 blue (water) and the normal rows in PF2 white (ice).

Decode the brick to see the method. 171 is %10 10 10 11: three brown brick pixels and a white mortar pixel. 255 is a full row of mortar, and 186 (%10 11 10 10) offsets the joint by two pixels for the next course of bricks.

4. The mode 5 version

For mode 5 the display list must be shorter: 12 rows of 16 scan lines fill the same 192 lines. Replace lines 50–60 with:

50 POKE 559,0:POKE DL+3,69
60 FOR I=DL+6 TO DL+16:POKE I,5:NEXT I
65 POKE I,65:POKE I+1,PEEK(560):POKE I+2,PEEK(561)

Line 65 writes the JVB (65) and the display list's own address straight after the twelfth row, so ANTIC never reaches the leftover mode 2 bytes. Only screen rows 0–11 are visible now, so move your drawing up. The characters come out twice as tall, which suits big title graphics and chunky platform scenery. Remember that the OS screen editor still believes it has 24 lines, so don't PRINT to a hand-built mode 5 screen; poke screen memory instead.

5. A metatile map in assembly

Real games rarely store a full 960-byte screen per room. They store a map of larger tiles ("metatiles") and expand it at run time. This MADS program defines its font in binary, so you can see the bit pairs, then expands a 20 × 12 map of 2 × 2-character metatiles into a mode 4 screen with its own display list:

; ANTIC 4 metatile map - mads tiles.asm -o:tiles.xex
SDMCTL = $022F
SDLSTL = $0230
COLOR0 = $02C4          ; PF0 shadow (708)
CHBAS  = $02F4
SCREEN = $4000
ptr    = $80            ; zero-page pointer to screen row
tmp    = $82

        org $2000
main    lda #0
        sta SDMCTL              ; screen off while we build
        ldx #4
setcol  lda colours,x           ; 708-712
        sta COLOR0,x
        dex
        bpl setcol
        lda #>font
        sta CHBAS
        jsr drawmap
        lda #<dlist
        sta SDLSTL
        lda #>dlist
        sta SDLSTL+1
        lda #$22                ; normal playfield, DMA on
        sta SDMCTL
        jmp *

; expand 20x12 metatiles (2x2 chars each) into 40x24 screen
drawmap lda #<SCREEN
        sta ptr
        lda #>SCREEN
        sta ptr+1
        ldx #0                  ; X = index into map
row     lda #20
        sta tmp                 ; metatiles left in this row
        ldy #0                  ; Y = column in screen row
cell    stx mapidx
        lda map,x
        tax                     ; X = metatile number
        lda mt_tl,x
        sta (ptr),y
        lda mt_bl,x
        pha
        lda mt_br,x
        pha
        iny
        lda mt_tr,x
        sta (ptr),y
        tya                     ; bottom row is 40 bytes further on
        clc
        adc #40
        tay
        pla
        sta (ptr),y             ; bottom right
        dey
        pla
        sta (ptr),y             ; bottom left
        tya
        sec
        sbc #38                 ; back to top row, next column
        tay
        ldx mapidx
        inx
        dec tmp
        bne cell
        lda ptr                 ; advance two screen rows (80 bytes)
        clc
        adc #80
        sta ptr
        bcc @+
        inc ptr+1
@       cpx #240                ; 20 x 12 metatiles done?
        bne row
        rts
mapidx  dta 0

; metatiles: 0 sky, 1 brick, 2 grass, 3 water (inverse = PF3), 4 tree
mt_tl   dta 0, 1, 2, 3+128, 6
mt_tr   dta 0, 1, 2, 3+128, 7
mt_bl   dta 0, 1, 4, 3+128, 8
mt_br   dta 0, 1, 4, 3+128, 9

colours dta $C6,$24,$0E,$84,$96 ; PF0 green, PF1 brown, PF2 white, PF3 blue, BAK

map     :5 dta 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
        dta 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
        dta 0,0,4,0,0,0,1,1,1,1,1,1,0,0,0,4,0,0,0,0
        dta 2,2,2,2,2,2,1,1,1,1,1,1,2,2,2,2,2,2,2,2
        dta 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
        :3 dta 3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3

        .align $400
font    dta 0,0,0,0,0,0,0,0             ; 0 blank (all background)
        dta %10101011                   ; 1 brick: 10=PF1 brick, 11=PF2 mortar
        dta %10101011
        dta %10101011
        dta %11111111
        dta %10111010
        dta %10111010
        dta %10111010
        dta %11111111
        dta %00010001                   ; 2 grass top: 01=PF0 green
        dta %01010101
        dta %01010101
        dta %10011001
        dta %10101010
        dta %10101010
        dta %10101010
        dta %10101010
        dta %11001111                   ; 3 water/ice: 11=PF2, or PF3 if inverse
        dta %11111111
        dta %11111111
        dta %11111100
        dta %11111111
        dta %11111111
        dta %00111111
        dta %11111111
        :8 dta %10101010                ; 4 earth: solid PF1
        :8 dta 0                        ; 5 unused
        dta %00000001,%00000101,%00010101,%01010111   ; 6 tree top left
        dta %01010101,%01110101,%01010101,%00010101
        dta %01000000,%01010000,%01010100,%01010101   ; 7 tree top right
        dta %01011101,%01010101,%01010100,%01010000
        dta %00000000,%00000000,%00000010,%00000010   ; 8 trunk left
        dta %00000010,%00000010,%00001010,%00101010
        dta %00000000,%00000000,%10000000,%10000000   ; 9 trunk right
        dta %10000000,%10000000,%10100000,%10101000
        .align $400                     ; rest of the 1K set stays zero

dlist   dta $70,$70,$70
        dta $44,a(SCREEN)               ; mode 4 + LMS
        :23 dta $04
        dta $41,a(dlist)

        run main

Points worth copying:

  • Binary dta lines make the font readable. %10111010 is visibly "brick, mortar, brick, brick". Many people design on squared paper with four columns per character, then type the bit pairs directly.
  • Four lookup tables (mt_tl, mt_tr, mt_bl, mt_br) give the characters of each metatile corner. Adding a metatile costs four bytes, and a room costs 240 bytes instead of 960.
  • The water metatile uses character 3 + 128, so the whole lake is drawn in PF3 while the same character elsewhere could be white ice.
  • .align $400 puts the font on a 1K boundary. The second .align simply pads the set to its full size so the display list doesn't overlap it.

The display list is three blank-8 instructions, $44 (mode 4 with LMS) and 23 more $04 bytes, then JVB ($41). For mode 5, use $45 and eleven $05 bytes. Or mix the two: a few mode 5 rows for a big status bar above a mode 4 playfield is a common layout.

A map also helps with game logic. To find what a sprite stands on, convert its coordinates to a map cell and look the tile up. That is quicker and more predictable than testing collision registers (see Collision Detection in Depth).

6. Designing a four-colour font

  • Think in wide pixels. A character is only four pixels across. Letters such as M and W are nearly impossible, so plan text as two-character-wide letters or put text in a separate mode 2 area (see Character Set Switching with DLIs).
  • Build objects from blocks. A 16 × 16-pixel object is a 4 × 2 block of mode 4 characters. Make each terrain type in several variants with the details offset slightly, and mix them so the screen doesn't look like a grid of repeated stamps.
  • Assign colours by role. Keep background and PF0/PF1 for colours shared across the whole screen, and use bit pair 11 for the "accent" colour that varies between PF2 and PF3 per character.
  • Budget the 128 characters. Character 0 should normally be blank, since cleared screen memory is full of zeros. If you need more than 128 shapes on one screen, a display list interrupt can switch fonts part way down.
  • Colours per line are not fixed. A DLI can rewrite COLPF0–3 and COLBK between rows. Many games change two or three colours every few rows to get far more than five colours on screen.

Tools help. In the 1980s COMPUTE!'s Second Book of Atari Graphics printed a "Four-Color Character Editor" and the "Fontbyter" screen designer, and Antic published Paul Chabot's "ANTIC 4/5 Editor Animator" (October 1984). Today, cross-platform editors such as Mode4 and Graph2Font export fonts and maps as assembler data.

Troubleshooting

  • Screen shows stripes of garbage: screen memory isn't cleared, or character 0 isn't blank.
  • Colours look right but shapes are scrambled: the font isn't on a 1K boundary, or CHBAS points at the wrong page.
  • The OS's text cursor or messages appear as coloured junk: that's normal in mode 4. Turn the cursor off (POKE 752,1) and avoid PRINTing.
  • Mode 5 screen rolls or flickers: the display list still contains 24 rows. Mode 5 needs 12 rows and a JVB after them.
  • Inverse characters look the same as normal ones: PF3 (711) is set to the same colour as PF2 (710), or your shape uses no 11 pixels.
  • Text window lines look odd after GRAPHICS 12 on an XL: the window is mode 2 and uses the same font. Use GRAPHICS 12+16 for a full-screen mode 4 display, or switch fonts with a DLI.

Further reading

  • COMPUTE!'s Second Book of Atari Graphics (COMPUTE! Books, 1984): Orson Scott Card's "The Four-Color Character Modes", the Four-Color Character Editor, and Fontbyter.
  • De Re Atari (Atari, 1982), Chapter 3, Figure 3-3: the colour-selection table.
  • Atari Graphics and Arcade Game Design by Jeffrey Stanton and Dan Pinal (Arrays, 1984), Chapter 3: multi-colour characters, with a worked four-character boat.
  • Antic, October 1984: "ANTIC 4/5 Editor Animator" by Paul Chabot.
  • Mapping the Atari by Ian Chadwick (COMPUTE! Books), Appendix 15: the XL/XE GRAPHICS 12 and 13 modes.
  • Altirra Hardware Reference Manual, section 4.4: character modes in hardware detail.

Credits