Burns kicked the English side 6-0 ahead at the Sportsground before James Simpson-Daniel and Connacht captain Gavin Duffy exchanged tries coming up to half-time.
The second period was bookended by penalties from Niall O'Connor and Burns as the Cherry and Whites stood firm to pick up only their second away victory in all competitions since February.
Connacht showed their intent in the opening minutes with Ethienne Reynecke snaffling a turnover and David Gannon bounding up the right wing.
Gloucester scrambled well though and despite struggling for early territory, Burns edged them ahead with an eighth-minute penalty from halfway.
Pressure from the restart led to Dario Chistolini infringing at a scrum, but O'Connor failed to find the target.
Burns then spotted a mismatch in midfield to storm into the Connacht 22, setting up a third penalty attempt which he converted.
Connacht were let down by a series of mistakes - loose lineouts being the most obvious - and Gloucester cranked up the pressure for a deserved 31st-minute try.
Their skipper Luke Narraway went close before play was swung out to the left, the Cherry and Whites biding their time until a neat offload out of the tackle from Eliota Fuimaono-Sapolu sent Simpson-Daniel over in the corner.
Burns failed to add the extras but the concession of the try prompted Connacht to lift their game.
O'Connor's high kick broke for Duffy to gather and slalom his way past Olly Morgan and glide over by the posts.
O'Connor converted to close the gap to 11-7 before Gloucester almost ended the half with a second try, Charlie Sharples just running out of space as he hurtled onto Henry Trinder's teasing grubber kick.
An early offside in the second half saw O'Connor make it a one-point game, slotting from the right.
A powerful couple of scrums and a lineout maul got Gloucester back into scoring range, but they turned down a kickable penalty and Connacht's committed defence kept them out.
However, it fell to Burns to punish Rodney Ah You for being caught offside on 72 minutes, his close-range penalty edging Gloucester four points clear, although the losing bonus point got Connacht off the mark in their first Heineken Cup campaign.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The entire All Blacks apparel line has been updated for 2013/14. Check out the New Zealand All Blacks polo.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Gilbert Blitz 8S rugby cleat is a great cleat at a great price of $69.99. Get a new pair of cleats today.
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
$29.99
The All Blacks Performance t-shirt is black with hints of blue from the training jersey. Very Cool.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.