Factual error: The Western Electric Model 639A microphone prominently featured in front of J Edgar Hoover in the 1934 Washington DC Senate appropriation hearing scene, wasn't invented till late 1939.
Continuity mistake: When John gives Billie her mink coat, the amount of wrap paper coming out of the corners of the box differs between the front and back angles.
Chosen answer: Baby Face Nelson doesn't pick up Dillinger. He picks up the characters played by Stephen Dorf and another associate. Dillinger and Hamilton escape through the woods and steal a car from an older man they come across in the forest. After the shootout, Dillinger and Nelson are never in the car together.