The Pro12 champions had to beat the four-time winners of the competition to keep pace with Leicester, who claimed a late 14-13 success over Treviso.
The Welsh region were always in front, courtesy of a Dan Biggar penalty and drop goal with 21 minutes played. Two Jean Marc Doussain efforts sandwiched another Biggar penalty as Ospreys led 9-6 at the break.
Doussain missed the chance to level the match with a penalty and that proved costly as Biggar padded Ospreys' advantage before Eli Walker scored the only try of the match in the 62nd minute to complete the scoring.
The Ospreys will need to beat Leicester and Treviso in their remaining two fixtures and hope other results go their way in order to advance.
Biggar had booted the home side into a third-minute lead before a loose pass from Justin Tipuric halted a promising attacking situation. He doubled the lead with a well-taken drop goal at the end of the opening quarter after another pinpoint up and under from the fly-half had caused panic in the visiting ranks.
The French side's increasing frustration showed when lock Patricio Albacete was sin-binned after using his feet to clear Ian Gough away from a rolling maul.
Doussain got Toulouse on the board in the 29th minute after more controlled mauling had put their hosts on the back foot.
The Ospreys twice turned down shots at goal in an effort to make their numerical advantage tell with a try but, after being repelled by some superb Toulouse defence, Biggar was soon asked to step up to make it 9-3.
Doussain hit back with a drop goal right on half-time but, with Wales tighthead Adam Jones now on the field for his first appearance since mid-October, the Ospreys remained in the ascendancy during the early stages of the second half.
Toulouse slowly began to work their way into the game and Doussain missed an opportunity to level matters as he struck the post with a penalty, while their anger increased further when a member of the Ospreys staff prevented Medard taking a quick throw.
The game then turned decisively in the home side's favour on the hour mark. Toulouse skipper Jean Boulihou was sin-binned for a high tackle on wing Walker, with Biggar slotting the resulting penalty.
Walker, who had looked a constant threat with ball in hand, then escaped some weak tackling down the left-hand touchline to finish off excellent Ospreys build-up play.
Biggar could not add the extras but the Welsh side were brimming with confidence, and only a wonderful cover tackle from Gael Fickou prevented Ashley Beck scoring under the posts after Richard Fussell and Andrew Bishop had got free thanks to the officials missing a blatant forward pass.
After their lung-bursting efforts the Ospreys tired as time began to tick down and Toulouse desperately sought a bonus point. The French pack could not ground the ball after rumbling over the line, while Ross Jones was sin-binned for illegally disrupting the maul.
A try looked like it had to come when Luke Burgess led a breakout with several men in support during the final breathless passage of play, but the Australian dallied and the final pass flew into touch as the Ospreys held out to breathe life back into their European campaign.
Gilbert has released a new line of rugby cleats. The Gilbert Virtuo 8S is part of the exciting new product. Check it out.
The entire All Blacks apparel line has been updated for 2013/14. Check out the New Zealand All Blacks polo.
The Nike Tiempo is a solid rugby cleat and one of few styles still made from full-grain natural leather.
The Gilbert Blitz 8S rugby cleat is a great cleat at a great price of $69.99. Get a new pair of cleats today.
A cool looking all black rugby cleat with the high performance adidas is known for. Get in the Gear!
Wear the crest of the British and Irish Lions on your t-shirt. A great look for the summer.
$29.99
The All Blacks Performance t-shirt is black with hints of blue from the training jersey. Very Cool.
The New Zealand All Blacks training jersey for 2013/14. Get in the Gear!
The USA Rugby Pro Alternate rugby jersey is perfect for any fan of the Eagles. Get yours to wear during the summer Test matches.
The NEW All Blacks 2013/14 jersey has arrived at World Rugby Shop. Dare to wear the colors of the All Blacks.