But the Saints prop is hopeful his side can use the Gloucester crowd to their advantage on Saturday.
Jim Mallinder’s men head to the west country sitting 12 points clear at the top of the Aviva Premiership.
But with Gloucester able to call on the likes of Wales star Richard Hibbard and England men Jonny May and Billy Twelvetrees, the home side will pose a stern test.
And Waller knows the Cherry and Whites supporters will also have a big say in deciding the outcome of this weekend’s encounter.
“They’ve got a massive pack and it’s a very hostile place to go with the Shed,” Waller said. “It’s a great atmosphere there and it’s somewhere I enjoy playing.
“It’s very similar to a hostile Welford Road when it’s full and that helps them.
“But it also adds a certain atmosphere to the game that pumps our boys up.
“It’s going to be a tough game and there’s no reason why we can’t win it if everything clicks, but we’re going to have to be at it.
“Gloucester are a decent side and their back three, especially, are very dangerous.
“Their pack are big and if they get on the front foot we’ll be in a bit of trouble so we’ve to to negate that straight away.”
With just six games of the regular season to go, Saints are in a strong position as they seek to secure a home play-off semi-final.
But Waller has warned that their sizeable lead can soon be eroded and he says the players are working hard to ensure that doesn’t happen.
“We’re happy that we’re 12 points clear but that can all change in a week or two,” he said.
“It’s massive for us to stay at the top because we want that home semi-final.
“We want to keep the gap as big as possible and there’s no reason why we can’t stay on top if we stay focused.
“The coaches and the players know that and it’s led internally by the senior players.
“There’s plenty of pride there and we don’t want to take our foot off the gas.
“Even though we’re winning a bit ugly at the moment, we’re working hard in training to keep that gap as big as possible.”