And in trying to firewall IPv6 I found t ... / 2007-10-26

And in trying to firewall IPv6 I found that INPUT and FORWARD are really separate. From the docs:

the built-in chains INPUT (for packets coming into the box itself), FORWARD (for packets being routed through the box), and OUTPUT (for locally-generated packets).

So if I want to set a policy for both the local machine and the machines behind it I need to set those rules in both INPUT and FORWARD.