The pilgrimage of St. Georgenberg is one of the most famous and ancient in Tyrol. So ancient, indeed, that its origin appears to have dated as far back as the end of the tenth century, when a chapel was consecrated here by Albuin, the then Bishop of Brixen. Even before this, however, Scherer asserts that a young Bavarian nobleman named Rathhold, from Aiblingen, "having learned the hollowness of the joys of even his great position, made up his mind to live apart from the haunts of men in some wilderness and solitary spot." And in pursuit of this determination he wandered on through the fertile fields and valleys of his own land and those of the Inn until he at length reached this spot in the Stallen valley, and ultimately came to the rock upon which the church stands. Up on the mountain-side he carved out for himself a cave where he lived as a hermit. But after a while a desire possessed him to go to some of the shrines of the greatest saints. He visited many, even travelling so far afield as to the shrine of St. Jago de Compostella; and at length returned once more to his hermit's cave to finish his days in prayer and contemplation. But he brought back with him a picture of the Madonna, over which with his own hands he reverently erected a protective shrine.
Soon from all the district round about, and even from distant parts of Tyrol and Bavaria, people came to worship at the shrine; and ere long "Our Beloved Lady under the Lindens" became a great pilgrimage resort. One day, years afterwards, so the story goes, there came to the place another young Bavarian nobleman who had wandered far in pursuit of game, and on hearing of the shrine had determined to visit it himself to ascertain what were the attractions and virtues of a place which was so venerated by the peasants of the mountains and valley round about. On his arrival at the little chapel he sought the hermit guardian, when what was his joy and astonishment to discover, in the white-bearded recluse, the elder brother whose strange disappearance from his castle home years before had caused much grief. Overjoyed at the meeting, the younger brother vowed that he would build a chapel on the spot more adequate for the protection of the holy and miraculous picture, and also a "shelter house" for pilgrims.