Niall O'Connor looked to have earned Connacht their first win of the competition when his 72nd-minute penalty put the visitors 19-16 up in the Pool Six clash at Kingsholm on Saturday, but replacement centre May then sailed past two defenders for the decisive score.
Tiernan O'Halloran and Tim Taylor had earlier traded first-half tries while O'Connor kicked four penalties and Taylor and Freddie Burns combined for three for the hosts.
May's late intervention was a bitter blow for the gallant Irish side, who were unlucky to lose after being the better team for much of the match against an out-of-sorts Gloucester.
The hosts lacked penetration and were behind for most of the match, too often kicking possession aimlessly away with Mike Tindall and Taylor the worst culprits.
Connacht made the better start and had the first opportunity for points, but O'Connor was short with a 55-metre penalty attempt.
Gloucester soon responded with an exhilarating run from James Simpson-Daniel and, when the Irish were penalised, Taylor kicked his side into a seventh-minute lead with a simple penalty.
However, Connacht continued to be the better side in the opening exchanges and stung Gloucester with the opening try of the game after 11 minutes.
Eliota Fuimaono-Sapolu's pass just inside his own half was intercepted by scrum-half Frank Murphy, who combined cleverly with Gavin Duffy and George Naoupu before O'Halloran collected the final pass for a superb try, which O'Connor converted.
Stung by the unexpected reverse, back came Gloucester. After a fumble from Duffy, Simpson-Daniel narrowly failed to win the approval of the television match official for a touchdown, but from the resulting five-metre scrum Taylor ran strongly to force his way his over for a try which he converted.
The visitors responded with an excellent counter-attack featuring powerful runs from Eoin Griffin and O'Halloran and, when the hosts were penalised, O'Connor tied up the scores after 30 minutes.
Some clever tactical kicking from Duffy and O'Connor kept Gloucester on the back foot and the Irish regained the lead with a penalty from O'Connor after 37 minutes to enable the visitors to hold a deserved 13-10 interval advantage.
Certain sections of the crowd were dissatisfied with the Gloucester performance in the first half and, on the restart, Burns was introduced at fly-half in place of Taylor.
But there was to be no quick remedy as once again Connacht began well and extended their lead with another O'Connor penalty.
Further home changes saw Rory Lawson and Akapusi Qera enter the fray and, spurred on by impressive efforts from Simpson-Daniel and Olly Morgan, there was more urgency in Gloucester's play and they brought the deficit back to three points with a penalty from Burns.
Connacht should have nullified that score but O'Connor missed with a 20-metre drop-goal attempt and the outside-half was soon made to pay for that error as a second penalty from Burns made it 16-16 with 11 minutes remaining.
Another fine run from O'Halloran gained the Irish a position in the home 22 and, when Gloucester were penalised, O'Connor fired over a straightforward penalty and a memorable victory for Connacht seemed likely until May's late effort.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The Barbarians are one of the top invitational rugby sides with a long history and classic rugby jersey.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Lions get ready for their matches with this green training jersey. It's what the players wear. Get in the Gear!
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
The Lions are ready to get their Australia tour underway. They arrive in Perth on Monday.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.