Arrow on the Doorpost - S3-E13
Continuity mistake: In the final scene of the episode, Rick and Hershel are talking while looking out over the prison yard. Near the end of the conversation, Rick squarely faces Hershel and says "I'm hoping you could talk me out of it." The camera then changes to Hershel, and Rick is now immediately facing 90° to his left looking over the yard. When the camera changes back to Rick, he is now looking directly at Hershel.
Answer: The walkers could have knocked them down more than once. There may have been another wave that knocked down the fence earlier and they're going to put it up again soon.
Captain Defenestrator