My Fair Ernest T. Bass - S4-E17
Other mistake: Barney says, "I brought the car to drive you over to Mrs. Wiley's," but later when Andy, Barney, Ernest T., and Ramona leave Mrs. Wiley's we see that they're at the house that's actually next door to Andy - though to hide this fact there's a station wagon in the driveway, and they've removed the trellis beside the door.
Answer: Those "boxes" are valences that used to be very common, before air conditioning. They allow for windows to be open during rain storms. They permit air circulation, without letting the rain in.