Although Berwick Preservation Trust has identified funding worth £3.6 million there is still a shortfall of over £1.5 million.
With costs rising by a further £20,000 a month, councillors have voiced concerns whether the scheme will ever get off the ground.
Coun John Robertson, speaking at a meeting of Berwick Borough Council, said: "I am starting to think we should give up on this project.
"The shortfall is almost 30% of the total costs and what is more we seem to be getting further and further away from the target.
"Every month that goes by adds £20,000 to the bottom line so if it's delayed by another six months we would have to find a further £120,000. That's a big worry."
The scheme aims to create a vibrant, multi-use facility incorporating an arts and exhibition area, a bistro including outside courtyard seating, meeting/seminar rooms and a 42 bed youth hostel.
However, the plans were dealt a big blow earlier this year when a major funding bid to the Community Assets Fund, administered by the National Lottery, was turned down.
Coun David Stewart said: "It seems to me that we are having to find more and more money to put in the pot and we're still no closer to seeing it come to fruition.
"I wonder why no-one has taken a bulldozer to it if we can't resolve this funding problem."
In an effort to plug the shortfall, a £1 million funding bid to the Sea Change initiative - Government money to regenerate coastal resorts - has now been submitted.
But Coun Robertson said: "I am led to believe this fund is only a £12 million pot for the whole country and we are looking for nearly 10% of that. Is that realistic?"
Despite these concerns, councillors agreed to provide £5000 as one-off revenue funding to Berwick Preservation Trust to enable it to prepare the funding bid and progress with work over the next six months.
Leader, Coun Isabel Hunter admitted: "It's been a struggle over the last few years but we're down to the last £1 million and we are getting there."
Coun Dougie Watkin added: "This has been a long drawn out process and there is no doubt that the granary looks like a bulldozer needs to be taken to it but we have a unique opportunity here where by putting in £5000 we could achieve a £1 million investment that would enable this project to go ahead."
Over £3.1 million has been secured so far with another £430,000 provisionally in place. Major funders so far include Northumberland Strategic Partnership with One NorthEast which have committed over £1.5 million. A further £500,000 has been secured from the Heritage Lottery Fund, £350,000 from the Northern Rock Foundation, £320,000 from Berwick Borough Council and £270,000 from the Youth Hostel Association.
The six-storey granary was built in 1769 between Bridge Street and the Quayside but has stood empty since 1985.