The visitors triumphed 23-21, courtesy of Leigh Halfpenny's 80th-minute penalty, which was awarded by English referee Wayne Barnes when Irish flanker Stephen Ferris tip-tackled Ian Evans.
Ferris received a yellow card, as did Wales lock Bradley Davies earlier in the game, although Davies' tackle on Donacha Ryan was far more serious and looks certain to see him cited and banned.
Wales outscored Ireland 3-2 on tries, and then Halfpenny came up trumps under colossal pressure to send his team on the march towards a possible Six Nations title and Grand Slam.
But Gatland said: "We were reasonably lucky. I thought we were only at about 70 per cent today.
"But to come here and be under pressure and to come back with 14 men shows great character.
"The displeasing thing was a bit of a lack of discipline with the yellow card. We got out of jail, but we've won our first game of the tournament away from home.
"We were down at half-time in the World Cup against Samoa, but we kept our patience and finished on top of them. That is what Test match rugby is all about - taking your opportunities."
Gatland felt Davies was lucky to only get a yellow card, and admitted he fears the worst in terms of a citing.
Wales are already without the services of injured locks Luke Charteris and Alun-Wyn Jones, so any period of suspension for Davies could hit second-row resources hard.
"We've got to plan, potentially, for him being cited," Gatland added.
"I've seen a replay, and I won't deny it wasn't fantastic. Potentially, we have got to prepare for the worst.
"When you go behind with 15 minutes to go and you have a yellow card, you don't have a lot of time to get back into the game.
"But this win has given us massive confidence, and we are dangerous when we are playing with belief and confidence. We know there is a lot of improvement in our game."
Gatland hailed a "world-class" performance by wing George North, who scored one of Wales' three tries, while he had sympathy for fly-half Rhys Priestland, who missed two easy penalty kicks.
"We shouldn't have put so much pressure on Rhys with the kicks. He has been out (injured) for a few weeks. It is such a confidence role, but I thought he was excellent in the second half."
Halfpenny took over kicking duties from Priestland, and he admitted his last-gasp strike as "the hardest kick I've had to take in all my life".
He said: "As soon as it went over the emotion came. Having to deal with the miss against France in the World Cup semi-final, putting that over makes it all worthwhile."
As for the game itself, Halfpenny added on BBC1: "It was end to end stuff, an exciting game. It was a tough test and all credit to Ireland, they really gave us a tough game.
"The score was back and forth, and I'm just really pleased to get the result for the rest of the campaign."
Wales captain Sam Warburton, meanwhile, could know in the next 24 hours whether or not he has a realistic chance of being fit for next Sunday's home game against Scotland.
Warburton went off at half-time nursing a dead leg, and he said: "I will go back and have treatment tonight and tomorrow, and the physios will make the call.
"The game reminded me a bit of the World Cup semi-final (Warburton was sent off). It's horrible being sat on the bench not being able to influence the game, but I am chuffed to bits.
"I think there is a lot more to come from this team."
Ireland captain Paul O'Connell believes his team only had themselves to blame after seeing their Six Nations Grand Slam and Triple Crown hopes shredded on the tournament's opening weekend.
The hosts led 21-15 when Tommy Bowe scored 12 minutes from time - a score that followed hooker Rory Best's earlier touchdown - but North's late effort after two Jonathan Davies tries gave Wales a dramatic victory.
"We gave Wales a lot of ball earlier in the game that we shouldn't have done," O'Connell said.
"We put ourselves in a position to win the game, and we conceded eight points in the last five minutes.
"We also conceded a lot of momentum in the first-half and gave them a lot of belief. We struggled to get into the game in the first half, and you can't give a team that length of time with ball in hand.
"Defensively, we will have some work to do. We conceded a lot of ground and yardage at times, but when we attacked, we played with intensity."
Asked about the late penalty award following Ferris' tackle, O'Connell added: "Stephen hasn't said a lot.
"I was right beside it, and I didn't think there was a problem straightaway, but I haven't seen it on video. I don't think that was the winning and losing of the game."
Ireland now have just a six-day turnaround before facing Six Nations title favourites France in Paris, and O'Connell said: "We have plenty of experience in the group, and we will recover to the best of our ability.
"We need to be patient and trust our defence."
Ireland coach Declan Kidney said: "Three teams have had their Grand Slams ended this weekend, and three teams are still in. That is the nature of it, but there is a championship still to be won.
"We will take a good look at what we can solve ourselves. We gave Wales some easy field position, and they made inroads.
"We had to defend for 60 per cent-plus of the game, and if you do that then you are going to ask for trouble. There are different aspects of the game I know we can improve on.
"They managed to put the ball wide on us, and we need to scan a bit better than we have been doing. It is just a case now of getting ready for the next match in six days' time."
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The entire All Blacks apparel line has been updated for 2013/14. Check out the New Zealand All Blacks polo.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Gilbert Blitz 8S rugby cleat is a great cleat at a great price of $69.99. Get a new pair of cleats today.
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
The All Blacks Performance t-shirt is black with hints of blue from the training jersey. Very Cool.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.