Deliberate mistake: There are several historical dates wrong in this episode. For a start, Prince George didn't become Prince Regent until 1811, and yet this is set before the Battle of Trafalgar in 1805 (as Black Adder gives Wellington some advice on it). Similarly, Wellington didn't get the title "Duke of Wellington" until 1814. Again inconsistent with being pre-Trafalgar.
Deliberate mistake: There are a lot of jokes with the name of the book "The Prince and the Pauper." However the book was published in 1881, 51 years after the Prince Regent died.
Answer: It's not known what Baldrick's first name is. When he was a kid and introduced himself as Baldrick to some other kids, they responded with, "We know. Sod off, Baldrick." Sod off is another way of telling someone to go away or get lost. So when the kids said "Sod off, Baldrick," what they were really saying to him was, "Get lost, Baldrick."