Trials (fitnah) do not cease coming over the heart unless the slave’s entire way of life is for the sake of Allah, lofty and sublime is He. Thus his love would be for Allah and for anything Allah loves and his anger would be for Allah and anything that angers Allah, and likewise his alliances and hostilities. Otherwise love for the creation will draw him in. Love of the creation is a means for him to attract them towards him, the one makes the other stronger. Thus when he defeats his whims, neither something defeated not its being beloved will attract him towards it despite his whims. On account of the defeat of his whims, there is a prohibitive factor for whims for his self, due to what is in his heart of fear and love for Allah which prevents him from gravitating toward beloved things.
As for people’s love for him, then indeed it necessitates that they draw him in, and they with their strength, towards them. So if he does not possess the strength, to repel them from his self out of love an fear of Allah, then they will draw him in and take him towards them, like the love of the wife of al-Aziz for Yusuf. For indeed the strength of Yusuf and his love, sincerity and fear for Allah was stronger than the beauty and charm of the wife of al-Aziz and his fondness for her. This is since one of them loved his appearance, and in addition to that, the motives were stronger than him and them. In this case the one who is protected is he who Allah protects. Otherwise, in most cases some evil happens between people regarding love from both angles.
Due to this, the Messenger of Allah, peace and blessing of Allah be upon him, said:
“A man is not alone with a woman except Shaytan is the third of them.” (Ahmad)
Gardens of Purification by Ibn Taymiyyah Pages 83-84