Visiting hooker Simon Alcott scored the only try of a cracking clash, but the highly-rated England prospect Burns punished the Chiefs as Gloucester came from behind to win by two points.
Exeter fly-half Gareth Steenson converted Alcott's try and added three penalties of his own but it was not enough.
Exeter arrived at Kingsholm buoyed by two excellent victories in the Premiership over the last month in claiming the scalps of both London Irish and Wasps that put them in a healthy mid-table position before kick-off.
The heat was on both sides as Gloucester were one place above the Chiefs in fifth and one point better off at the start, having beaten Sale Sharks last week.
The sell-out crowd were not disappointed either as both teams got to grips with each other at a frantic pace right from the start.
Burns struck first with a 30-metre attempt before Steenson landed a booming 35-metre effort from the left touchline and then put the Chiefs into a 6-3 lead with an easy 15-metre shot after an infringement by the Cherry and Whites.
Exeter had the most territory in the opening half-hour as their front five, lead by props Brett Sturgess and the huge tight-head Hoani Tui, thumped into the Gloucester midfield as scrum-half Haydn Thomas scampered the ball away at the breakdowns.
Gloucester found their way back into the match as the interval approached and nearly got the first try of the match when full-back Rob Cook played a delicate chip-and-chase over the head of the Exeter defence and, after pressure on the visitor's line, scrum-half Jimmy Cowan was held-up over the line.
Referee Llyr Ap Geraint Roberts brought play back for a Gloucester penalty which Burns booted to level the scores as tempers flared in the last seconds of the half, leading to a lecture for both captains.
It may not have been the high-scoring Premiership fiesta of last season at Kingsholm where the Cherry and Whites came home 37-23 but it was intense and engaging.
Burns put Gloucester six points ahead with two penalties after the teams resumed, much to the disgust of furious Chiefs coach Rob Baxter.
Exeter would not lay down, though, and, after Steenson landed a third penalty of the match, the Chiefs put pressure on the home side which eventually told as the Gloucester defence was breached as Alcott picked up a ball from a ruck and just got over the line to touch the ball down.
Steenson converted but he lead was cut soon afterwards to a single point as Burns landed his fifth penalty to set up a thrilling finale.
And fatally, Exeter went over the top of a ruck with seven minutes left, leaving Burns to boot his final penalty to put the Cherry and Whites back in front and see them to victory.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The entire All Blacks apparel line has been updated for 2013/14. Check out the New Zealand All Blacks polo.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Gilbert Blitz 8S rugby cleat is a great cleat at a great price of $69.99. Get a new pair of cleats today.
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
The All Blacks Performance t-shirt is black with hints of blue from the training jersey. Very Cool.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.