Thank you everyone for your replies! After talking it out, we've actually decided to go with a trip to Washington D.C. I've always loved museums and my husband loves politics and history so it's a perfect fit for this trip. I love all of the recommendations for VA too so I'm thinking maybe we'll make that another trip later in the year!
We stayed at Embassy Suites in Crystal City. It was very clean and had a nice breakfast and happy hour. It was nice to have two rooms. That would be ideal with the baby. They have microwaves and refrigerators in the rooms which I always looked for with a little one. It was a little farther out which made it a bit more affordable. It was an easy walk to the metro which would great for commuting with a stroller or Uber worked well also.