Chris Boyd's men fell 21-0 down at Kingsholm and eventually lost 31-21 in the Challenge Cup last-16 clash at Kingsholm.
There was no shortage of spirit as Saints battled hard with Alex Coles in the sin bin and Dan Biggar off the field after seeing red in the 52nd minute of the match.
But they ultimately fell short as Gloucester booked their place in the quarter-finals.
"It was a very frustrating night and it's definitely an opportunity missed," Ludlam said.
"We left ourselves with way too much to do after the first 25, 30 minutes.
"I can't fault the fight but we need to come out of the blocks a bit faster.
"It's the story of this team at the moment that we've got heaps of fight when it gets tough, but we need to come out of the changing room way faster.
"We need to have that mentality that we're going for them straight away because it's knockout rugby.
"We just need to get stuck in right from minute one."
And Ludlam added: "We're always going to make chances because we've got some unbelievable attacking threats when we've got the ball.
"It's just about keeping teams down to a minimum of points - that's our big work-on.
"We've got to be stronger around our own 22 and our try line because, at times, we're way too easy to score against.
"The good thing now is that we can put all our eggs into one basket and that's getting top four in the Premiership."