Walk with Me - S3-E3
Revealing mistake: Merle is sitting in a chair and talking with Michonne and Andrea. At one point he takes off the prosthetic on his right arm to show them what used to be his hand (it's only a nub now), but shortly afterwards you can see his right hand in a fist as he puts the prosthetic back on. (00:13:00)
Answer: The walkers could have knocked them down more than once. There may have been another wave that knocked down the fence earlier and they're going to put it up again soon.
Captain Defenestrator