Chris Boyd's side had moved into a 22-14 half-time lead thanks to a Tommy Freeman double and a score from scrum-half Alex Mitchell
They did it despite losing three players to the sin bin, just as they had during their incredible win at the Dragons a week earlier.
But this time Saints couldn't get the job done as, despite having 15 men for the full second 40, they were eventually outgunned by Ulster.
The men from Belfast scored two tries in the final eight minutes to earn a 35-27 win and book their place in the semi-finals.
Saints were 27-21 up when Ollie Sleightholme added to their tally with a try in the 59th minute, but Ulster kept fighting and got the win they desperately wanted.
Saints had initially picked up where they had left off against the Dragons as Teimana Harrison and Taqele Naiyaravoro carried hard to take the hosts within range, allowing Mitchell to pick up and score intelligently.
Piers Francis was on kicking duties and he slotted the conversion to give his side a 7-0 lead inside seven minutes.
Naiyaravoro was forced off for a head injury assessment due to his hard work in the build-up for Saints, bringing Ahsee Tuala into the action.
Saints soon lost Nick Isiekwe to the sin bin for a late tackle on Billy Burns and it was to get worse almost immediately as Sam Matavesi was yellow carded for side entry and collapsing an Ulster maul.
Referee Alexandre Ruiz, who was also in charge at Rodney Parade last weekend, headed under the posts to award the penalty try and Saints had a huge spell ahead with just 13 men, just as they experienced against the Dragons.
But Saints responded well as Alex Waller won a big breakdown penalty, and his side eventually landed three points to add to their tally and waste some of the sin-bin periods.
However, after Isiekwe returned, Saints lost another player to the bin, as David Ribbans was yellow carded.
Ulster took advantage after a lengthy spell of pressure, with Rob Herring eventually powering over as the relentless pressure finally told.
John Cooney converted to make it 14-10 to the away side, but Saints refused to be downbeat about having a man fewer and they scored through Freeman.
Francis missed the conversion but it took just one more minute for Freeman to score again as he finished off a scything breakaway move that involved Harrison and Mitchell.
Francis made no mistake with the conversion and Saints led 22-14, enjoying the novelty of having 15 men on the field.
Ulster tried to come out firing at the start of the second half and after being held up over the line, they eventually forced their way over through tighthead prop Marty Moore.
Cooney converted to cut the gap to a single point and Saints suffered a big blow when Freeman was forced off injured, having to be helped from the field after seemingly jarring his leg.
But Saints kept pushing on and after another mighty carry from Lewis Ludlam, Tuala did superbly to offload to Sleightholme, who scored out wide.
Francis missed the conversion to leave the gap at six points with 20 minutes of the match remaining.
But Ulster were soon ahead, piecing together a fine try that resulted in Cooney stretching out an arm to score.
The scrum-half also converted to put his team one point up and Ulster rarely looked in danger after getting a point up.
They just built on their lead as Jacob Stockdale was released for a score, and Cooney converted to deliver an eight-point lead.
And that was to be that for Saints as they bowed out after an enthralling encounter at the Gardens.
Saints: Freeman (Dingwall 55); Sleightholme, Hutchinson, Francis, Naiyaravoro (Tuala 7); Biggar, Mitchell (James 75); Waller (cc) (Auterac 67), Matavesi (Haywood 67), Hill (Painter 67); Ribbans, Coles (Moon 70); Isiekwe (Wood 73), Ludlam (cc), Harrison.
Ulster: Lowry; Baloucoune, Hume (Mathewson 79), McCloskey, Stockdale; Burns (Madigan 75), Cooney; O'Sullivan (Warwick 67), Herring, Moore (O'Toole 58); O'Connor; Treadwell, Reidy (Rea 52); Murphy (c), Timoney.
Referee: Alexandre Ruiz