Bury picked up their first win of the season thanks to a late winner from Danny Swailes after Swansea had fought back from a 2-0 interval deficit.
Defender Swailes bundled in a shot from team-mate Jon Newby ten minutes from time after he'd previously headed an own goal to bring the Swans level.
"I thought I'd let myself and everyone else down when it went to 2-2," admitted the Shakers centre-half who gave keeper Glynn Garner no chance with a back-header off a long throw by substitute John Williams.
And in a frenetic finish the visitors' Steve Watkin was shown the red card by Hull referee Boyeson for his off the ball fracas with Swailes' defensive partner Steve Redmond.
Bury played their best football for many a month in the opening 45 minutes and deserved to be two goals clear.
George Clegg rifled in his second of the campaign via a post to give Bury a ninth minute lead and Newby with his first goal since last March doubled the advantage following a good run and cross by Lee Unsworth.
Swansea looked ragged at the back and it was a surprise when manager Nick Cusack used his three substitutes, but didn't bring off limping goalie Roger Freestone.
Just after the hour the Welshmen were thrown a lifeline by a linesman. Redmond appeared to have taken the ball cleanly off the toe of James Thomas but a penalty was given.
Garner saved Thomas's spot-kick but could do nothing to stop the Swansea marksman netting at the second bite.
As Bury became anxious within sight of the winning post Williams' howitzer from the touchline skimmed off Swailes' closely cropped head into the back of the goal.
"I wondered which way we would go but I'm proud of the lads for the character and the bottle they showed to win the game," said manager Andy Preece.
"I thought we had players out there who were living on past reputations," said Swansea chief Cusack.
"At times they looked like non-league players and I've got to think seriously about the contribution they're making to this team."