Sharks skipper David Seymour crashed over the line in the first half and a conversion and a penalty from fly-half Nick MacLeod helped the hosts into a 10-0 half-time lead.
But two penalties from Tim Taylor got Gloucester off the mark after the restart and a late try from centre Henry Trinder, which Taylor failed to convert, set up a grandstand finish.
Sale scrum-half Dwayne Peel, who has enjoyed a fine start to the season after being overlooked by Warren Gatland for Wales' World Cup squad, was ruled out due to a shoulder injury.
His place went to former Leeds scrum-half Scott Mathie, but Gloucester made a vibrant start to force Sale back into their own 22.
The visitors' backline is renowned for its pace and willingness to run the ball at every opportunity, but Gloucester's pack were also prominent in the opening exchanges.
The hosts were afforded the chance to get off the mark in the fifth minute, but MacLeod saw a penalty from inside his own half fall just short.
Nevertheless, the pendulum soon swung in Sale's favour and a superb kick from wing Joaquin Tuculet gave them the field position to pressure Gloucester's line.
That pressure told in the 15th minute when Sam Tuitupou drove at the heart of the Gloucester defence from deep inside the 22.
His progress was halted but he regained possession and slipped a pass into Seymour to crash over from close range for a try which MacLeod converted.
Five minutes later, Taylor missed a relatively straightforward penalty after Sale incurred the wrath of referee Sean Davey.
Neither side was able to find much fluency before half-time as try-scoring opportunities remained few and far between.
MacLeod chipped a teasing kick into the left corner shortly before the break and Gloucester full-back Olly Morgan dealt with the danger, only to slide into the advertising hoardings and sustain a hand injury.
He was immediately replaced by Freddie Burns, but moments later MacLeod kicked a penalty to put Sale 10-0 up at the break.
Taylor then kicked a penalty to get Gloucester up and running two minutes after the restart before Sale forward Henry Thomas was sinbinned and Taylor kicked his second penalty.
MacLeod kicked a penalty for Sale on the hour but nine minutes later Gloucester fashioned a well-worked score for Trinder, who burst clear inside the left channel.
However, Taylor could not convert and Sale held on during a tense finale to maintain their fine start to the campaign.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The Barbarians are one of the top invitational rugby sides with a long history and classic rugby jersey.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Lions get ready for their matches with this green training jersey. It's what the players wear. Get in the Gear!
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
The Lions are ready to get their Australia tour underway. They arrive in Perth on Monday.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.