Swans boss Brian Flynn blasted his side for making schoolboy errors as his side failed to beat struggling Southend who ended the game with just eight men.
The Shrimpers had Dave McSweeney, Jay Smith, Mark Dower, and assistant manager Paul Brush all banished from the pitch by referee George Cain but Flynn's men couldn't stop themselves going down to their second home defeat of the season.
"We have to take it on the chin," said Flynn. "But that performance wasn't good enough and the players can do better both individually and as a team.
"To say we were poor was an understatement and we didn't defend well as a team after they had a player sent off.
"We made schoolboy errors at crucial times and we got punished for that. The old saying is that you only get what you deserve and we didn't deserve to win that." Flynn saw his side throw away a one goal lead after James Thomas opened the scoring on 14 minutes with a well directed header.
But it took just three minutes for Leon Constantine to level the scores and Gower had caretaker boss Steve Tilson jumping for joy with his side's second on the stroke of half time.
And despite a second half onslaught from the hosts, substitute Mark Warren sealed the vital win in the dying stages.
The late goal was made all the more sore for Swansea fans after Smith and Gower had been sent off five minutes earlier, joining McSweeney in the early bath after he was sent off after just 20 minutes for kicking out at Andy Robinson.
And Thomas' late second was no more than a consolation score to leave Flynn with a lot to think about before next week's tricky tie at Doncaster.