Tweak serialization of ScorerUsingTime for better forward compat