It was Rangers' second cup exit in just over a week - they also lost to Queen of the South in the Challenge Cup - but they will have taken heart from their showing at Somerset Park.
Berwick lost four times to Ayr in the Second Division last season but a bright opening saw them go ahead inside 12 minutes. David Greenhill, making his first start of the season, was the player who gave them the lead.
Gradually Ayr started to get into their stride and as the half progressed they began to exert more pressure on the Rangers' back four of Guy, Ewart, Kiczynski and McMahon.
It looked as if they might hold out to half-time, but two goals just before the break swung the match the Honest Men's way.
They equalised after 40 minutes through Chris Aitken. This followed a spell of sustained pressure, and then, almost right on the half-time whistle, they took the lead through Alex Williams, who netted from close range after a good build-up.
In the second half the heavy rain made it difficult for both sides but the young Berwick side, still finding its feet after being completely rebuilt, held their own and Greenhill saw a shot scrambled away with the defence out of position.
The superior experience and strength of Ayr enabled them to control the closing stages but Rangers looked much more secure with the back four looking confident.
Berwick - McGurk, Guy, Ewart, Kiczynski, McMahon, Little, Greenhill, Craig Anderson (Callaghan 59), Dillon (Howat 83), Lister, Gribben (Grant 79). Subs (not used): McLean, Lunn.