Continuity mistake: When Phyllis first visits Walter's apartment she said he forgot his hat. When you watch the scene as he's leaving her house, on the way to the door he picks his hat up from a small table and puts it on.
Suggested correction: This is not an error - Walter is being openly facetious about him forgetting his hat.