Don't remove nodes if there's no channel_update for a temp failure