Alfreton had the better of the first half and Dan Bradley smacked a 30-yard effort off the crossbar before the hosts took the lead on 19 minutes when Todd Jordan headed home a corner.
Jamie Jackson sent an effort wide for the visitors while Cleveland Taylor had a free-kick saved by Matt Duke before the break.
But the Steelmen worked their way back into it in the second half and only a last-ditch challenge from Joe Leesley denied Nathan Evans.
And, after Ryan Robbins had seen a free-kick blocked, the equaliser arrived at the death when Milnes finished things off after the ball had fallen into his path.
AFC Rushden & Diamonds’ title hopes suffered a setback as second-placed Kings Langley claimed a deserved 1-0 victory at the Dog & Duck to close the gap to just one point at the top of the Evo-Stik Southern League Division One Central.
Diamonds turned in a poor display in front of a crowd of 613 as they created few clear-cut chances.
The only goal of the game came on four minutes when Ryan Plowright beat Matt Finlay with a low shot after good work from Steve Ward.
From then on, Kings Langley remained solid and had the best chance in the rest of the game when Mitchell Weiss scooped over the bar when he should have done better.
Despite the loss, Diamonds remain top of the table but Kings Langley are just a point adrift with a game in hand.
Kettering Town’s home clash with Dunstable Town in the Evo-Stik Southern League Premier Division was called off at 1.30pm.
The Latimer Park pitch was deemed unplayable by the referee following this morning’s rain.