The Cherry and Whites had looked well on course for victory after opening up a 23-10 advantage late in the first half thanks in part to a Tom Voyce score and a penalty try.
But Irish, who had claimed the opening try through Alex Gray in the first minute, hit back as Topsy Ojo and Steve Shingler went over to help them establish a 27-23 lead.
Hooker Darren Dawidiuk responded with the hosts' third try while Tim Taylor kicked 15 points, but it was a penalty from Burns which eventually sealed the win after Exiles full-back Tom Homer, who racked up 15 points of his own, had tied the scores with his third penalty.
Gloucester made the worst possible start when fly-half Taylor's attempt to launch a counter-attack saw his loose pass picked off by Gray inside 60 seconds, giving the former Newcastle man a simple canter to the posts for a try converted by Homer.
Taylor made some amends for his error by putting the Cherry and Whites on the board with a fourth-minute penalty, before the hosts crafted a beautiful try.
Centre Henry Trinder cut through the Exiles midfield defence and kept his composure to move the ball on to Olly Morgan, who sent Voyce haring in at the left-hand corner, with Taylor adding a testing conversion and a penalty soon after for a 13-7 lead.
Homer had been off target with his second and third penalty efforts, but he landed his fourth to trim the gap to three points after 24 minutes.
However, Taylor quickly added his third penalty before the Exiles saw Chris Hala'ufia sin-binned.
The Tongan number eight was fortunate not to see red after a reckless high shot on Morgan, and his moment of madness proved costly as Irish's creaking scrum found it impossible to stem the tide a man down, conceding a penalty try when Gray kicked the ball away with the Gloucester pack rumbling over the line.
Irish refused to buckle and they responded in devastating fashion with 17 points in just eight minutes either side of the break.
Firstly, Ojo raced 40 yards up the middle to finish brilliantly from Dan Bowden's slick inside pass, with Homer converting to have them just six points adrift at the interval.
The England Under-20s cap cut the gap to three with a penalty early in the second half before Adam Thompstone stripped Charlie Sharples of the ball to set a position for centre Shingler's superb angle of running to send him over for his first Irish try, Homer again added the extras for a 27-23 lead.
But their advantage did not last long as Gloucester went straight up the other end for hooker Dawidiuk to crash past some hesitant defence for a converted score.
Back came the visitors with Thompstone only being denied a try by a desperate covering tackle from Sharples after more sharp work by the lively Joseph, with the pressure creating another penalty chance which Homer gratefully received to tie the scores.
Replacement Burns snuck Gloucester back in front with a 63rd-minute penalty, but his failure to land a simple chance late on could have proved costly had Ojo not denied what would have been a winning try as Bowden's chip ahead bounced away from the winger in the dying moments.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The entire All Blacks apparel line has been updated for 2013/14. Check out the New Zealand All Blacks polo.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Gilbert Blitz 8S rugby cleat is a great cleat at a great price of $69.99. Get a new pair of cleats today.
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
The All Blacks Performance t-shirt is black with hints of blue from the training jersey. Very Cool.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.