The outside-half kicked 17 points to seal a game which neither side deserved to lose.
The visitors will reflect that Ryan Lamb failed with two simple penalties, which deprived the Saints of victory - and they will rue the carelessness which gave away a vital try just before the interval.
The home backs were sharper with fine displays from Burns and Jonny May but in the second half they were deprived of possession by a formidable Northampton pack that grew stronger as the game progressed, with Roger Wilson and Soane Tonga'uiha at the core of that effort.
Gloucester began strongly with two defence-splitting runs from Eliota Fuimaono-Sapolu but casual passing prevented the hosts opening their account.
However their opening try was not delayed for long, with Charlie Sharples outflanking the Saints' defence for the score - which Burns converted for a sixth-minute lead. The visitors' poor start continued when young wing, Jamie Elliott departed with an injury to be replaced by Tom May.
Northampton responded with a simple penalty from Lamb but that was soon nullified by one from Burns.
Against the run of play, the Saints should have scored their opening try but Courtney Lawes, returning from a six-week injury absence, lost the ball in the process of diving for the try-line and that error was further compounded when former Gloucester man Lamb missed with a simple 22-metre penalty.
In quick succession, Burns and Lamb exchanged penalties as the Northampton pack became the more dominant force, which enabled Lamb to kick another goal to leave the Saints only three points adrift
However on the stroke of half-time, Northampton carelessly gave away possession in their own 22, allowing May and Sharples to counter-attack - and Darren Dawidiuk was on hand to take the pass and score a crucial try which gave Gloucester an 18-9 interval lead.
Gloucester fumbled the restart kick and at the resulting scrum gave away a penalty - but once again Lamb failed to take advantage as he fired his 25-metre kick wide.
After 50 minutes Lamb was successful with another chance as the Saints continued to dominate the opening stages of the second half but their backs lacked the cutting edge of their home counterparts.
Burns extended the home lead with his third penalty before the Saints introduced another former Gloucester player in prop Paul Doran-Jones, with Lamb also being substituted in favour of Stephen Myler.
The Saints declined to take two kickable penalties in favour of more attacking options and were rewarded when, from a lineout five metres out, Tonga'uiha drove over for the try for Myler to convert to bring Northampton within two points.
Another long range penalty from Burns temporarily calmed the Gloucester nerves but the Saints broke from halfway for George Pisi to score an excellent try. Myler was unsuccessful with the conversion to leave the scores level with 11 minutes remaining.
Back came Gloucester, Burns' angled 25-metre penalty attempt rebounded back off a post but the outside-half was given another opportunity when the Saints again offended and this time he made no mistake with a simple kick to give his side a narrow victory.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The Barbarians are one of the top invitational rugby sides with a long history and classic rugby jersey.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Lions get ready for their matches with this green training jersey. It's what the players wear. Get in the Gear!
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
$29.99
The Lions are ready to get their Australia tour underway. They arrive in Perth on Monday.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.