Olver had been assured with the boot in the 24-16 victory at Newcastle six days earlier and he was on the money once more in a scrappy 19-13 success.
Charlie Clare grabbed his first try for Saints, and their only one of the match, as Jim Mallinder’s men saw off a sloppy Gloucester side.
It was a game lacking quality and the Cherry and Whites fought back with late tries from Ollie Thorley and Dan Thomas, adding to Billy Burns’ first-half penalty.
But Saints managed to close out the win, which was their third in succession as they prepare to head back to the Premiership with a trip to Worcester Warriors next Friday.
It was a second victory in the space of little more than two weeks for Saints against Gloucester, who had suffered a narrow 23-20 loss in Northampton in the Premiership last month.
The Cherry and Whites included Salesi Ma’afu in their side on this occasion and his first act on his return to the Gardens was to knock on, bringing cheers from the Gordon Terrace.
But Saints lost the resulting scrum, giving Gloucester a penalty and a chance to build, but they failed to do so, letting their hosts off the hook.
And Saints escaped again soon after as James Wilson failed to clear, running into traffic close to his own line, but Gloucester’s Lewis Ludlow knocked on at the crucial moment.
Burns did put the away side ahead soon after though, making the most of a scrum penalty to register three points.
Saints responded with a scrum penalty of their own as Olver slotted an easy kick from in front of the posts.
It was a scrappy game, with Gloucester kicking for territory astutely and Saints failing to string attacks together at the other end, with attempted moves just not clicking.
But Olver landed another penalty to put the home team ahead and they were soon cranking up the heat in the forwards, sending hooker Clare over for his first Saints score.
OIver converted to make it 13-3 and Burns missed a long-range penalty before the break to ensure the score stayed that way at half-time.
Gloucester were soon down a man in the second period as flanker Thomas paid the price for persistent infringing from the away side.
Saints finally registered some points as Olver landed a place-kick after Gloucester No.5 Joe Latta had baited the fans before giving away a penalty by running into contact.
Latta was replaced soon after, bringing loud pantomime-style cheers from the Saints supporters, who were craving any sort of entertainment.
Thomas returned from the sin bin with Gloucester having only conceded three points while he was off the field, but Saints had a healthy 16-3 lead going into the final 25 minutes.
Olver soon made it 19-3 with a tidy drop goal and Gloucester continued to squander any sort of chance they got at the other end, with knock-ons blighting the away side’s chances.
But they finally got a try as wing Thorley found himself in space and managed to get over the line.
Full-back Lloyd Evans missed the conversion to leave Saints 11 points up, with 14 minutes remaining,
Things got unexpectedly nervy late on as Thomas picked up a loose ball to score, but Evans again failed to add the extras and Saints closed the game out to earn the win.
Saints: Wilson; Elliott, Burrell (Foden 61), Hutchinson, Collins; Olver, Kessell (Dickson 55); E Waller (Beesley 63), Clare (Fish 60), Hill (Denman 63); J Onojaife, Craig; Gibson (Nutley 60), Ludlum, Dickinson (c) (Harrison 53).
Gloucester: L Evans; Protheroe (Vellacott 70), Purdy, Creed, Thorley; Burns (Hudson 58), Braley; Y Thomas (Hohneck 53), Walker (Mullis 61), Ma’afu (Knight 40); Batley (Beckett 42), Latta (Clarke 53); Ludlow (c), G Thomas, G Evans.
Referee: Craig Evans