Signed Numbers on the 6502: Two's Complement, Overflow and Signed Comparisons
Sooner or later every Atari game needs negative numbers: a player moving left, a ball whose vertical speed changes sign when it bounces, a sine table for a wave effect. The 6502 has no separate "signed" instructions, but its ADC, SBC and flags handle negative numbers perfectly well once you know the convention, two's complement, and which flag to look at. This guide explains both, corrects a common misunderstanding about the overflow flag, and finishes with tested routines for signed comparison, sign extension and halving, used in a small bouncing-ball simulation.
What you'll learn: how two's complement works and why the same ADC serves signed and unsigned numbers, what C, V and N mean for each, how to negate 8- and 16-bit values, how to compare signed numbers correctly, sign extension, arithmetic shift right, and signed fixed-point motion.
What you need: MADS and an emulator, and the basics from Getting Started with 6502 Assembly on the Atari 8-bit. Integer Maths on the 6502: Multiply, Divide, BCD Scores and Printing Numbers covers unsigned multiplication and division. The example was assembled with MADS and run in atari800, and every result shown below was read off its screen.
1. Two's complement
A byte holds 256 patterns. Unsigned, they mean 0 to 255. Signed, the patterns from $80 to $FF are reused for −128 to −1:
| Byte | Unsigned | Signed |
|---|---|---|
| $00 | 0 | 0 |
| $7F | 127 | +127 |
| $80 | 128 | −128 |
| $FB | 251 | −5 |
| $FF | 255 | −1 |
Bit 7 is the sign, but the rest of the byte is not simply the size of the number. −5 is not $85. It is the value that gives zero when added to +5, ignoring the carry out of bit 7: $FB + $05 = $100. To negate a number, invert every bit and add one:
lda #5
eor #$FF ; $FA (one's complement)
clc
adc #1 ; $FB = -5
Atari Roots walks through why inverting alone (one's complement) gives wrong sums, and why adding the one fixes it. The practical point is this: because negative numbers are defined so that ordinary binary addition works, the 6502 uses exactly the same ADC and SBC for signed and unsigned numbers. Only your interpretation of the result, and of the flags, changes.
2. What the flags mean
After ADC or SBC, the CPU sets every flag as if both interpretations were possible. You pick the flags that match the one you mean:
- C (carry) is the unsigned overflow: set if an addition passed 255, or clear if a subtraction needed to borrow. For signed numbers it means nothing on its own.
- V (overflow) is the signed overflow: set if the true signed result does not fit in −128 to +127. That happens when two numbers of the same sign are added and the result's sign comes out different.
- N (negative) is simply bit 7 of the result, the sign, which is correct only if V is clear.
The example's first three tests show the difference (results printed as result, C, V):
| Operation | Result | C | V | Meaning |
|---|---|---|---|---|
| −5 + 8 ($FB + $08) | $03 | 1 | 0 | +3: correct; the carry is irrelevant |
| 100 + 50 ($64 + $32) | $96 | 0 | 1 | signed overflow: 150 doesn't fit, $96 reads as −106 |
| −100 − 50 ($9C − $32) | $6A | 1 | 1 | signed overflow again: −150 doesn't fit |
A common misunderstanding
Atari Roots tells readers to use the overflow flag "rather than the carry flag to carry numbers from one byte to another" when adding signed numbers. That is not how multi-byte arithmetic works. A 16-bit signed number is simply a 16-bit two's-complement number, and its bytes are chained with the carry, exactly like unsigned numbers. V matters only after the last (highest) byte, where it tells you whether the whole result overflowed. Test 9 in the example adds $00F0 (240) and $FFE0 (−32) with an ordinary CLC, ADC, ADC sequence and gets $00D0 (208), the correct answer.
V has two other quirks. There is CLV but no "set V" instruction, although BIT sets V from bit 6 of the byte it tests. And V is changed only by ADC, SBC, BIT, CLV, PLP and RTI, never by CMP.
3. Comparing signed numbers
CMP is designed for unsigned comparison. It sets C if A ≥ the operand, as unsigned numbers. Test 6 shows the result for $FD (−3) against 2: carry set, "greater or equal", which is wrong for signed numbers. CMP also leaves V alone, so you can't fix the result afterwards.
The standard method is to do a real subtraction with SBC and look at N EOR V. If there was no overflow, N is the true sign of A − B. If there was, the sign came out inverted. So "A < B (signed)" is exactly "N ≠ V". The example's routine turns that into the carry flag:
scmp2 sec ; signed A < 2 ? returns C=1 if so
sbc #2
bvc sc1 ; no overflow: N is the true sign
eor #$80 ; overflow: the sign came out inverted
sc1 asl ; move the corrected sign bit into carry
rts
Test 5 prints 01 for "−3 < 2". We also checked the N EOR V rule against all 65,536 pairs of signed bytes with a short script, with no exceptions. Another well-known trick is to flip bit 7 of both values with EOR #$80 and then compare them unsigned. It works because it maps −128…+127 onto 0…255 in order.
4. Widening and halving
Sign extension. To turn a signed byte into a 16-bit value, the new high byte must be $FF for a negative number and $00 for a positive one, never just 0. Test 7 turns $FD into $FFFD. Forgetting this is the classic bug when an 8-bit speed is added to a 16-bit position: moving left becomes a jump of almost 256 to the right.
Arithmetic shift right. LSR divides an unsigned number by two, but it shifts a 0 into bit 7, which turns a negative number positive. To halve a signed number, the sign must be copied in instead. The 6502 has no instruction for that, but two do the job:
cmp #$80 ; carry = bit 7 (the sign)
ror ; shift right, copying the sign back into bit 7
Test 8 halves −7 ($F9) and gets −4 ($FC). The result rounds down (towards minus infinity), not towards zero, so −1 halved stays −1. The 16-bit version does CMP #$80 on the high byte, then ROR high and ROR low.
Branch offsets are signed bytes too. A branch can reach 128 bytes back or 127 bytes forward from the following instruction. While writing this example, MADS rejected two branches as "out of range" when the bounce code grew. The fix is a branch with the opposite condition over a JMP.
5. Signed motion: a bouncing ball
Games rarely need signed multiplication, but they constantly add signed velocities to positions. Fixed point makes that smooth. The example keeps the ball's height y and its vertical speed vy as 16-bit 8.8 numbers: the high byte is whole screen rows, the low byte is 1/256ths of a row. Each step:
vy = vy + gravity(gravity is $0030, about 0.19 rows per step per step).y = y + vy, an ordinary 16-bit addition. When the ball rises, vy is negative and the addition makes y smaller, with no special code.- At the floor, negate vy with a 16-bit
0 − vy, then remove a quarter of it, using two 16-bit arithmetic shifts and a subtraction, so each bounce is lower.
The trail of asterisks in the screenshot is the result: three bounces of decreasing height across the 40 columns.
6. The complete program
; Signed arithmetic on the 6502: test results printed as hex, then a
; bouncing-ball trail computed with a signed 8.8 fixed-point velocity.
SAVMSC = $58
PTR = $CB ; screen pointer
RES = $CD ; scratch
org $3000
start lda SAVMSC
sta PTR
lda SAVMSC+1
sta PTR+1
; 1. -5 + 8: signed result 3, carry set (ignore it), no overflow
clc
lda #$FB ; -5
adc #$08
jsr flags ; show result, C and V
; 2. 100 + 50: 150 doesn't fit in -128..127 -> V set
clc
lda #100
adc #50
jsr flags
; 3. -100 - 50: -150 doesn't fit either -> V set
sec
lda #-100
sbc #50
jsr flags
; 4. negate 5: invert all bits, add one
lda #5
eor #$FF
clc
adc #1
jsr hexa
; 5. is -3 < 2? signed compare: subtract, then N EOR V
lda #$FD ; -3
jsr scmp2 ; carry = 1 if A < 2 (signed)
lda #0
rol
jsr hexa
; 6. the same with an unsigned CMP: $FD >= 2, so carry is set
lda #$FD
cmp #2
lda #0
rol
jsr hexa
; 7. sign-extend -3 to 16 bits
lda #$FD
jsr sext ; high byte in A, low byte kept in RES
jsr hexa
lda RES
jsr hexa
; 8. arithmetic shift right of -7: -4 (rounds down)
lda #$F9
cmp #$80 ; copy the sign into carry...
ror ; ...and shift it back in
jsr hexa
; 9. 16-bit: $00F0 + (-32 = $FFE0) = $00D0; the carry chains the bytes
clc
lda #$F0
adc #$E0
sta RES
lda #$00
adc #$FF
jsr hexa
lda RES
jsr hexa
jsr bounce
jmp *
scmp2 sec ; signed A < 2 ? returns C=1 if so
sbc #2
bvc sc1 ; no overflow: N is the true sign
eor #$80 ; overflow: the sign came out inverted
sc1 asl ; move the corrected sign bit into carry
rts
sext sta RES ; sign-extend A: returns high byte in A
and #$80
beq sx1 ; positive: $00
lda #$FF ; negative: $FF
sx1 rts
flags php ; print A, then C and V as 0/1
jsr hexa
plp
php
lda #0
rol ; C into bit 0
jsr hexa
plp
lda #0
bvc fl1
lda #1
fl1 jsr hexa
jmp nextl
hexa pha ; print A as two hex digits plus a space
lsr
lsr
lsr
lsr
tax
lda hexd,x
jsr out
pla
and #$0F
tax
lda hexd,x
jsr out
lda #0
out ldy col
sta (PTR),y
inc col
rts
nextl lda #0 ; next screen line (only used after flags)
sta col
clc
lda PTR
adc #40
sta PTR
bcc nl1
inc PTR+1
nl1 rts
; ---- bouncing ball: y and vy are signed 8.8 fixed point ----
bounce lda #0
sta x
sta y
sta y+1
sta vy
sta vy+1
step clc ; vy = vy + gravity
lda vy
adc #$30
sta vy
lda vy+1
adc #0
sta vy+1
clc ; y = y + vy (an ordinary 16-bit add)
lda y
adc vy
sta y
lda y+1
adc vy+1
sta y+1
cmp #13 ; hit the floor (row 13)?
bcs floor
jmp plot
floor lda #13
sta y+1
lda #0
sta y
jsr negvy ; vy = -vy
lda vy+1 ; vy = vy - vy/4 (keep 3/4 of the speed)
sta t+1
lda vy
sta t
jsr asr16
jsr asr16
sec
lda vy
sbc t
sta vy
lda vy+1
sbc t+1
sta vy+1
plot lda y+1 ; screen row 10 + y
clc
adc #10
tax
lda SAVMSC
clc
adc rowlo,x
sta PTR
lda SAVMSC+1
adc rowhi,x
sta PTR+1
ldy x
lda #$0A ; '*' in screen code
sta (PTR),y
inc x
lda x
cmp #40
beq done
jmp step
done rts
negvy sec ; 16-bit negate: 0 - vy
lda #0
sbc vy
sta vy
lda #0
sbc vy+1
sta vy+1
rts
asr16 lda t+1 ; t = t / 2, keeping the sign
cmp #$80
ror t+1
ror t
rts
col dta 0
x dta 0
y dta a(0)
vy dta a(0)
t dta a(0)
hexd dta d'0123456789ABCDEF'
rowlo :24 dta <[#*40]
rowhi :24 dta >[#*40]
run start
The four result lines at the top of the screen read, in order: tests 1–3 (result, C, V on each line), then test 4 (FB), test 5 (01), test 6 (01), test 7 (FF FD), test 8 (FC) and test 9 (00 D0).
Troubleshooting
- An object moving left jumps to the far right: an 8-bit signed value was added to a 16-bit one without sign extension.
- Comparisons fail only for large values: you used CMP/BCC on signed numbers. It works only while both values have the same sign. Use SBC and N EOR V.
- A negative speed halves to a large positive one: you used LSR instead of
CMP #$80/ROR. - Results off by one after negation: the
CLCbeforeADC #1is missing, orSECbeforeSBC. - Overflow detection never triggers: V is only meaningful after ADC or SBC on the top byte. CMP doesn't change it.
Further reading
- Mark Andrews, Atari Roots (Datamost, 1984): chapter 10 on assembly language maths, and the description of the processor status flags.
- Avery Lee, Altirra Hardware Reference Manual, chapter 3: the 6502's flags.
- On this site: Integer Maths on the 6502: Multiply, Divide, BCD Scores and Printing Numbers, 6502 Addressing Modes, the Atari Way, Writing a Small Complete Game in Assembly: Catch!.
Credits
- Picture: The example program: the signed-arithmetic test results at the top, and a bouncing ball computed with a signed fixed-point velocity below. — Program authors; screenshot by the site
- Mark Andrews, Atari Roots (Datamost, 1984) — chapter 10 Signed Numbers (one's vs two's complement, overflow flag remark), chapter 3 status flags — Mark Andrews; AtariArchives.org, 1984
- Altirra Hardware Reference Manual — chapter 3 CPU flags — Avery Lee (Altirra)
