Bandit Bandit (Extended Version)