Scarlets recovered from 17-3 down at Sandy Park to get back on level terms early in the second half, but a late try from James Scaysbrook sealed victory for the Chiefs.
Leinster's losing bonus point in their game at home to Clermont denied Exeter second place in Pool Five, but they could still reach the later stages of the Amlin Challenge Cup while Scarlets' European season is over.
However, Baxter's immediate concern is their upcoming Aviva Premiership matches with west country neighbours Gloucester and Bath.
"The next two Heineken Cup games [away to Clermont and home to Leinster] can go to bed for a while now because we have some very important Premiership rugby to take care of," said Baxter.
"If we don't take care of that we would be in danger of letting the season fall flat a little because we were are in a nice place now, ticking along quite well but showing some consistency and improvement in some areas but getting a little dented in others.
"But as long as we keep fighting to take those steps forward then I will be very happy. We have backed up a win in the Heineken Cup with a second win and this was our first in front of our own supporters.
"I am confident that we can play better, which I have said to the players is a nice place to be in.
"There was a lot of endeavour out there and a lot of hard work. Sometimes you just have to graft your way through to a win and that is what we did."
Exeter went into the break 20-13 ahead after tries from centre Ian Whitten and hooker Simon Alcott, with fly-half Gareth Steenson converting both and adding two of his three penalty attempts.
Hooker Ken Owens crossed for the visitors only try of the half which fly-half Aled Thomas converted as well as landing two penalties.
The visitors were then back on level terms early in the second half with a try from Scott Williams converted by Thomas, but Steenson nudged them back in front with a penalty before converting skipper James Scayesbrook's late touchdown.
Baxter added: "We got off to a flying start playing some good pressure rugby, good hard carries, good positional kicking, pressure in the set piece and pressuring their kickers.
"We got in some good positions which we got one very good try from and from pressure we created a second.
"We were on form then and playing to a very strong game plan but then went a little off plan, but we tend to do that when we get a lead and get excited."
This was the first of five games on the road for Scarlets, and head coach Simon Easterby knows they must find some improvement in the way they start games before they face Cardiff Blues on Friday night.
Easterby said: "The game got away from us in the opening 20 minutes and you can't give any team two soft tries but that is what we did.
"The charge down was one which they didn't have to work for and then they scored off a set piece.
"When you gift them 12 points in the first half then it is always going to be an uphill struggle, and we did that last week as well when they got into an early lead.
"Our defence on the whole was good and we came back well in the second half. But when you give sides soft tries it is difficult to take that.
"I can't really put my finger on why we started slowly and we have had a couple of them recently. I think it needs to be addressed before we go in to the Blues match."
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The Barbarians are one of the top invitational rugby sides with a long history and classic rugby jersey.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Lions get ready for their matches with this green training jersey. It's what the players wear. Get in the Gear!
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
The Lions are ready to get their Australia tour underway. They arrive in Perth on Monday.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.