I don't find it offensive, but I can't tell you how many times I've been tempted to comment to people about the baby care centers. They're inside air conditioned (or heated if need be). Not as much commotion. I often see women nursing right near the centers and just want to tell them about the wonderful area they could go. I have yet to do it however, because I'm afraid they will think I am offending them. If you are near the babycare centers I would definately go use them. Also for changing a baby. Almost all the bathrooms (including the men's rooms) have changing tables, but the ones in the centers are nice and easier to use in my opinion.
If you're not near the baby care center, and feel comfortable in public anywhere is fine.