RUT 906 (and other models?): Failover not working as it should! RUT9M_R_00.07.07.1

Are you sure you configured Failover correctly?
Means interfaces set into correct order and both activated for Failover?
Nicely described here:

How to configure WAN failover on new firmware? - #3 by Chase