Other mistake: In Woodbury, on the street, while Andrea talks to the Governor, we can see everybody wearing jackets and orange, red and yellow trees, showing that the scene was shot during fall. According to the beginning of the third season, they had just passed through a hard winter. So it should be spring or the beginning of summer.
Other mistake: When Martinez shoots the grenade launcher for a second time the cross hairs are on the concrete wall of the guard tower yet the explosion does nothing to the concrete and destroys the top of The Tower.
Answer: The walkers could have knocked them down more than once. There may have been another wave that knocked down the fence earlier and they're going to put it up again soon.
Captain Defenestrator