Wellingborough Prison opened as a borstal in 1963 and held young offenders until 1990, when it became a prison for Category C adult men.
The Ministry of Justice announced last month that the prison, in Millers Park, Doddington Road, was to be closed by the end of the year.
The move will reduce the prison service’s capacity by 588 places.
Clive Bevan lives in Millers Park, the residential area where the prison is based, and worked as the prison’s gardener from 1988 to 2001.
He said: “The average day was getting to work for 7.30am, the inmates would come out at 8.30am then I would set them all doing different jobs – mowing the fields, mowing the gardens, doing the lines on the football pitches, weeding the gardens, planting plants at the proper time.
“We were growing different sorts of flowers, we used to grow geraniums and other bedding plants that went to Bedford prison and other prisons in the area.
“We had small vegetable patches and some went into the kitchens to help with the inmates’ diets.”
Mr Bevan, who has lived in Millers Park for 24 years, said he had fond memories of his time at the prison.
He said: “We did lots of different things. I had 21 prisoners come to work every day. It was an excellent job, I really enjoyed it.
“I enjoyed nearly every day, there weren’t many days I didn’t enjoy going to work. Part of it was that I was doing something I love doing, not only working with the inmates but doing the veg growing as well.”
David Palethorpe was part of the management committee at the prison until he retired in October 2011. Mr Palethorpe worked as head of learning, skills and employment and head of offender management during his eight-and-a-half-years at the prison.
Mr Palethorpe said the prison changed a lot during his time there.
He said: “Some prisons have a very similar type of population that keeps coming through, for instance the number of life sentences doubled in the time I was there.
“When I first went there in 2003 we only had 40 foreign national prisoners, it was then designated as a place for foreign national prisoners and we ended up having 100.”
Mr Palethorpe said another major change had been more women guards coming into the prison.
He said: “We had a major change, we saw an increase in the number of female officers and a decrease in the number of male officers – it was almost 50/50 by the time I left. That made a major difference.
“That made it a more humane place, I think, it became a more humane place to work.”
In the last two years Mr Palethorpe worked at the prison it was the overspill prison for Pentonville and Wandsworth prisons in London.
Mr Palethorpe worked at the prison under previous governors Jim Lewis and Peter Siddons. He said: “The two governors we had were absolutely exceptional in saying ‘the nature of prisons has changed, we have got to change with it and make sure we stop this reoffending’.
“They were both exceptional leaders. It was quite an interesting place to work. The staff there are fantastic.”
Mr Palethorpe said some of the projects carried out at the prison during his time there included inmates making benches for schools and birdboxes for pocket parks in the prison workshop.
Inmates would also work to adapt bicycles to be used in Africa and do up old wheelchairs to be sent overseas for use.
Wellingborough Council leader Paul Bell said over the years HMP Wellingborough had been and continued to be a large part of the local economy, and local people were now concerned about what would happen next.
He said: “It’s an important part of the borough. I think people have been shocked, people have been wondering what’s going to happen to the site afterwards, and it’s the loss of income and jobs. It’s a knock-on effect for the town.
“I think a lot of people are still in a state of shock. It was announced on the last day of Parliament, a lot of people have been away on factory fortnight, I don’t think it has hit them just yet.”
Wellingborough MP Peter Bone estimates 600 people are employed by the prison either directly or indirectly by local businesses and has been fighting for its survival.
Mr Bone has secured an emergency debate on the matter on September 5, when Parliament reconvenes following its summer recess.
Speaking in Parliament on the day the announcement was made, Mr Bone said: “What is most disturbing about the matter is the number of jobs that will be lost in my constituency.
“Up to 600 people might lose their employment, whether they are employed directly through the prison or indirectly through local businesses.
“The independent monitoring board annual report states that the prison improved from a level 2 to a level 3 prison.
“Wellingborough prison has moved from 123rd out of 130 in the prison rankings to 93 owing to the hard work and commitment of its governor and staff.
“It has also become far more cost-effective, with five per cent efficiency savings in 2011-12 and further planned efficiency savings of three per cent projected for 2012-13.
“This is a good, improving prison that is now being closed without any consultation or appeal process.
“Closure would have significant ramifications for my constituents.
“The prison management and officers have done everything they were asked to do and more.”