Deliberate mistake: Leaven spends too much time thinking about if 645 and 372 are prime numbers to be believable. But she knows the factors of 649 and the result of 26 to the power of 3 almost immediately. Plus anyone with any maths knowledge whatsoever would know that any even number (or one divisible by 5) can't be prime, so 645 and 372 could be dismissed without even thinking about it.
Continuity mistake: In the opening, the camera zooms in on an eye that is a very light greyish blue. The camera then cuts to a man lying on the floor. The implication is that it's this man's eye. However, the man's eyes are brown.
Plot hole: Kazam is supposed to be a math genius, something that the plot hinges on, but he makes several mistakes when calculating the number of prime factors to find out whether a room is trapped or not. He says that 462 has three prime factors, when it has four, that 206 has four when it only has two and that 563 has two and 911 has three when both are actually prime numbers.
Revealing mistake: When they get Renn back into the room after he has been shot in the face with something acid-like, the smoke comes from behind his head. Not his face.
Other mistake: Toward the beginning, when the characters wonder aloud what the rumbling noise is, someone suggests that it's the ventilation system, to which Rennes responds "No vents." However, in the passages between rooms there are panels with ventilation slits in them (which you can plainly see even as Rennes says the line), and it's pretty impossible that the characteristically level-headed and observant Rennes wouldn't have noticed that.
Continuity mistake: Throughout the movie they are told to suck on the buttons from their shirts. In one scene, once Holloway climbs into the next room, Quentin is sucking on his button. He immediately speaks to Worth, muddling some words because of the button in his mouth. All of a sudden, the button disappears from his mouth and is in his hand again, and he then puts it back into his mouth.
Continuity mistake: Throughout the movie, Leaven's glasses have a crack in them that appears and disappears.
Revealing mistake: When Kazan lowers himself into the blue sound-activated room that is rigged with metal spikes, the set cube wobbles from left to right, spoiling the illusion that the cube is made out of solid metal.
Continuity mistake: The blood patterns on Worth's face periodically change throughout the rest of the movie after Quentin beats him with the shoe.