Undocumented 6502 Instructions on the Atari: LAX, SAX, DCP and Friends

Atari text screen listing nine undocumented instructions with their results: LAX A, X: 5A 5A; SAX MEM: 30; DCP MEM, Z: 0F 02; ISB A: 1A; ASO A: 83; ALR A: 07; ANC A, C: 80 01; SBX X: 20; NOP ABS A: 11

The 6502 has 151 official opcodes, but a byte can hold 256 values. Most of the remaining opcodes don't simply fail. Because of the way the chip decodes instructions, they switch on two internal operations at once and do something useful, or at least repeatable. Demo coders and some game programmers use these "illegal" or undocumented instructions to save bytes and cycles. This guide explains which ones are safe, what the useful ones do, how to write them in MADS, and the one big compatibility catch on the Atari.

What you'll learn: stable, unstable and lock-up opcodes, the combined read-modify-write instructions, LAX and SAX, the immediate-mode oddities, multi-byte NOPs, MADS's mnemonics, and when not to use any of them.

What you need: MADS and an emulator or a real machine with a stock 6502 (Sally) CPU. The test program was assembled with MADS and run in atari800, which emulates the NMOS 6502's undocumented behaviour.

1. Stable, unstable and fatal

The unofficial opcodes fall into three groups:

  • Stable: these behave the same on every NMOS 6502. They include the combined instructions in section 2, LAX, SAX, ANC, ASR/ALR, ARR and SBX, and the extra NOPs.
  • Unstable: results vary between chips or with temperature. The manual names $8B (ANE), $93 (SHA) and $AB (LXA). The last has been reported to behave differently on an Atari 800 from what the formula predicts. Don't use these.
  • KIL ($02, $12, $22, $32, $42, $52, $62, $72, $92, $B2, $D2, $F2): the CPU locks up and ignores interrupts until reset. You will meet these mostly when a program runs off into data.

The Atari's 6502C "Sally" is an NMOS 6502 with extra halt logic, so it behaves like any other NMOS 6502 here.

2. Combined read-modify-write instructions

Six instructions perform a memory read-modify-write (like INC or ASL) and then use the result in an ALU operation on A:

MADS name Other common name Does
DCP DCM DEC memory, then CMP A with the new value
ISB INS, ISC INC memory, then SBC the new value from A
ASO SLO ASL memory, then ORA it into A
RLN RLA ROL memory, then AND it into A
LSE SRE LSR memory, then EOR it into A
RRD RRA ROR memory, then ADC it to A

They take exactly as many cycles as the read-modify-write instruction alone, so the second operation is free. They support the same addressing modes as the memory instructions, plus (zp),Y and (zp,X), which INC and ASL don't have. ISB and RRA follow the decimal flag, because they contain SBC and ADC.

Typical uses: DCP counter followed by BNE decrements a counter and compares it with A in one go. ISB steps a table entry and subtracts it. ASO/LSE shift a byte in memory and merge it into A, which is handy when building pixel or sound masks.

3. Loads, stores and immediates

  • LAX loads the same value into A and X and sets N and Z. It is useful for table lookups where both registers need the value. Modes include zero page, absolute, (zp),Y and abs,Y.
  • SAX stores A AND X to memory without changing any flags. It masks and stores in one instruction, without disturbing A.
  • ANC #imm is AND, but bit 7 of the result is also copied into the carry. One use is to clear the carry for free before an ADC, when you know bit 7 will be zero.
  • ALR #imm (ASR in some assemblers) is AND followed by LSR A.
  • SBX #imm (AXS) sets X = (A AND X) − imm, with carry set as for CMP and no borrow in. With A = $FF it subtracts a constant from X in one instruction: useful for stepping an index down by 8 or 16.
  • The multi-byte NOPs read an operand and do nothing. $0C is a 3-byte "NOP abs". Placed in front of a two-byte instruction, it makes code that falls through skip that instruction, while a branch to the instruction itself still executes it. That is the job the old BIT abs trick does, but without changing any flags.

4. The test program

Each test runs one instruction on known values and prints the outcome. The expected values are in the comments.

; Undocumented NMOS 6502 instructions: each test prints its result in hex.
SAVMSC  = $58
PTR     = $CB
ZP      = $CD               ; scratch byte

        org $3000
start   ldx #0
        jsr label           ; LAX: load A and X at once
        lda #$5A
        sta ZP
        lda #0
        ldx #0
        lax ZP              ; A = X = $5A
        jsr hexout
        txa
        jsr hexout

        ldx #1
        jsr label           ; SAX: store A AND X, no flags changed
        lda #$F0
        ldx #$3C
        sax ZP              ; ZP = $30
        lda ZP
        jsr hexout

        ldx #2
        jsr label           ; DCP: DEC memory, then CMP A with it
        lda #$10
        sta ZP
        lda #$0F
        dcp ZP              ; ZP = $0F, and A = ZP so Z = 1
        php
        lda ZP
        jsr hexout
        pla                 ; the Z flag (bit 1 of P)
        and #$02
        jsr hexout

        ldx #3
        jsr label           ; ISB: INC memory, then SBC it from A
        lda #$05
        sta ZP
        lda #$20
        sec
        isb ZP              ; ZP = $06, A = $20 - $06 = $1A
        jsr hexout

        ldx #4
        jsr label           ; ASO (SLO): ASL memory, then ORA into A
        lda #$41
        sta ZP
        lda #$01
        aso ZP              ; ZP = $82, A = $01 OR $82 = $83
        jsr hexout

        ldx #5
        jsr label           ; ALR (ASR): AND immediate, then LSR A
        lda #$FF
        alr #$0E            ; ($FF AND $0E) >> 1 = $07
        jsr hexout

        ldx #6
        jsr label           ; ANC: AND, with bit 7 copied into carry
        lda #$FF
        clc
        anc #$80            ; A = $80, C = 1
        php                 ; (printing changes the carry)
        jsr hexout
        plp
        lda #0
        rol
        jsr hexout

        ldx #7
        jsr label           ; SBX: X = (A AND X) - immediate
        lda #$FF
        ldx #$30
        sbx #$10            ; X = $30 - $10 = $20
        txa
        jsr hexout

        ldx #8
        jsr label           ; NOP abs: skips the next two bytes
        lda #$11
        dta $0C             ; NOP abs ...
        lda #$99            ; ...swallows this LDA #$99 as its operand
        jsr hexout          ; A is still $11
        jmp *

label   txa                 ; row X+1: print 14-character label X
        clc
        adc #1
        tay
        lda SAVMSC
        clc
        adc rowlo,y
        sta PTR
        lda SAVMSC+1
        adc rowhi,y
        sta PTR+1
        txa                 ; label offset = X * 14
        asl
        sta col
        asl
        asl
        asl
        sec
        sbc col
        tax
        ldy #0
lb1     lda labels,x
        sta (PTR),y
        inx
        iny
        cpy #14
        bne lb1
        sty col
        rts
hexout  pha                 ; two hex digits and a space at (PTR),col
        lsr
        lsr
        lsr
        lsr
        jsr dig
        pla
        pha
        and #$0F
        jsr dig
        inc col
        pla
        rts
dig     tay
        lda hexd,y
        ldy col
        sta (PTR),y
        inc col
        rts
col     dta 0
hexd    dta d'0123456789ABCDEF'
labels  dta d'LAX     A, X: '
        dta d'SAX     MEM:  '
        dta d'DCP   MEM, Z: '
        dta d'ISB       A:  '
        dta d'ASO       A:  '
        dta d'ALR       A:  '
        dta d'ANC    A, C:  '
        dta d'SBX       X:  '
        dta d'NOP ABS   A:  '
rowlo   :12 dta <[#*40]
rowhi   :12 dta >[#*40]
        run start

Every line on screen matched the prediction: LAX loaded $5A into both registers, SAX stored $F0 AND $3C = $30, DCP decremented $10 to $0F and set Z because A held $0F, ISB gave $20 − $06 = $1A, ASO shifted $41 to $82 and ORed it to $83, ALR gave $07, ANC returned $80 with the carry set, SBX left $20 in X, and the NOP swallowed the LDA #$99 so A was still $11.

Two things caught us out while writing it, and both are general lessons:

  • The ANC test first reported the carry as 0. The print routine runs between the instruction and the test of the carry, and its LSR instructions overwrite C. Saving the flags with PHP right after the instruction fixed it. When testing flags, capture them immediately.
  • MADS needs its own names for these instructions. aso, rln, lse, rrd, sax, lax, dcp, isb, anc, alr, arr, ane, anx, sbx, las, sha, shs, shx, shy, npo and cim are its list, so slo from another assembler's source must be renamed to aso. We checked the encodings in the MADS listing against the opcode list in the hardware manual: lax zp = $A7, sax zp = $87, alr # = $4B, anc # = $0B. Recent MADS versions warn when an unstable opcode is used. For a NOP with an operand, the example simply uses dta $0C.

5. The compatibility catch

The undocumented instructions exist only on the NMOS 6502. None of them work on the 65C02. On the 65C816 every opcode is a defined instruction, so an "illegal" opcode does something completely different there. CPU upgrade boards that put a 65C816 in an Atari (popular for speed and extra memory) will therefore crash or misbehave with any program that relies on these instructions.

Some practical rules follow:

  • For a demo or a size-coding competition on a stock machine, they are fair game and widely used.
  • For a game or utility that should run everywhere, avoid them, or keep them out of anything a 65C816 user would run.
  • Never use the unstable ones ($8B, $93, $AB and the other SHx/LAS forms), even on stock hardware.
  • Document them in your source. A reader, or you in a year's time, will not recognise $0C or sbx at a glance.

Troubleshooting

  • The program locks up completely, even RESET on a 400/800 doesn't help: a KIL opcode was executed, usually by running into data. Power-cycle (or reset an XL/XE).
  • Works in one emulator, not another, or not on real hardware: you used an unstable opcode.
  • Works on a stock machine, crashes with a 65C816 board: undocumented opcodes. Replace them with official sequences.
  • MADS reports an illegal instruction: use MADS's mnemonic (for example aso instead of slo), or emit the byte with dta.
  • ISB or RRA give odd results: decimal mode is on. Clear it with CLD.

Further reading

  • Avery Lee, Altirra Hardware Reference Manual, chapter 3: the full opcode table, sections 3.5–3.7 on undocumented instructions and 65C02/65C816 differences.
  • The MADS manual: the list of supported illegal opcodes.
  • On this site: MADS (Mad-Assembler): A Practical Guide for Atari Programmers, Fast Memory Fills and Block Moves in 6502 Assembly, Signed Numbers on the 6502: Two's Complement, Overflow and Signed Comparisons.

Credits