The visitors shocked the home side by taking the lead in the 24th minute through skipper Terry Skiverton and they went further ahead on the hour with a shot from Aidan Downes.
Although Andy Robinson pulled a goal back in the 78th minute, the Swans were unable to take any points from the game.
In the final minute, Darren Pratley was dismissed for bringing down Yeovil substitute Lloyd Owusu.
The Swans showed three changes from the side that won at Gillingham with Kevin Austin, Tom Butler and Febian Brandy all making the starting line-up.
The Swans were in complete control in the opening quarter but, although they won several corners, they were unable to trouble the home defence.
In Yeovil's first attack of the game Marvin Williams made space down the right before crossing to the far post where Skiverton rose unmarked to head home to give his side a shock lead.
Neither side made changes during the interval, but Roberto Martinez must surely have urged his players to put a little more vigour and passion into their performance for the second half.
But a hesitant home defence left Downes unmarked and he put his side two up with a half-volley struck from 25 yards.
The introduction of Paul Anderson pepped up the home attack and Robinson was on hand to rifle home a loose ball following an Anderson corner.
Anthony Barry missed an open goal in the 86th minute and four minutes later Pratley was dismissed, but the final whistle blew with both sides having cause for celebration as Swansea had claimed the title while Yeovil were now safe from relegation.