The Holy Ancestors of Rasool Allah^-saww, Bani Hashim, were the custodians of the Keys to the Holy Kabah, so that is why they were referred to as Alay-Allah.

وانتزع علم الكهنة، وبطل سحر السحرة، ولم تبق كاهنة في العرب إلا حجبت عن صاحبها، وعظمت قريش في العرب، وسموا آل الله عزوجل

(Imam From Abu Abdullah Al-Sadiq^-asws having said in a lengthy Hadith) And the knowledge of the sooth-Sayers was snatched away, and the sorcery of the sorcerers was invalidated. And there did not remain a sooth-sayer among the Arabs except that he hid himself from his companion. And the Quraysh were considered great, and they (Bani Hashim^-asws) were called progeny of Allah^-azwj (آل الله) Mighty and Majestic’.

قال أبو عبد الله الصادق عليه السلام: إنما سموا آل الله لانهم في بيت الله الحرام

Abu Abdullah Al-Sadiq^-asws said: ‘But rather, they were named as progeny of Allah^-azwj (آل الله) Mighty and Majestic because they were in the Sacred House of Allah^-azwj’.^[1].[2]

[1] Bihar Al-Anwaar – V 15, The book of our Prophet^saww, Ch 3 H 9

[2] ) AAMALI-E-SADOOQ.Page-285: Bihar.Vol-15, Page-257: Rawzatul Wayazeen.Vol-1,Page-66