However, many, many people were never told that by Disney cast members. They used the system as it was described in all official brochures, on the website and even on the back of the ticket.
I have been going multiple times every year for over 30 years and every year since Fastpass began, not once did a CM volunteer that info to anyone in our group. Had I not read about it on here, I would have never known.
I am curious when you got your Fastpass did they just come up to you and say "oh by the way, you don't have to really come back at the time it says", or did they tell you that when you asked if you could come back late.
But as you said, it does not matter, or actually it did matter to Disney because they stopped it.