As has been previously mentioned, one should make an effort to dress neatly in the presence of others, no matter who they are. When visiting parents, pious people, elders, a relative, a friend, no matter how close, we should maintain cleanliness and elegance not ugly unsightliness.
Humans are naturely attracted or repulsed by what they see. Those that look good in clean clothes, smell nice, and are pleasant to look at people are attracted to them and enjoy their presence. Those that adopt the opposing characteristics, often cause harm to those that they meet with, through their physically unsightly appearance, their odour or otherwise. Hence this should be avoided to prevent people looking down on you (no matter who they are, or how close they are).
So to look neat and presentable when visiting someone or being visited is an Islamic manner. Do no ignore this aspect because you consider yourself close to those you are with. So if we should be visited whilst in casual clothing, we should change for the visitor, in order to enhance respect and complement your hospitality. These were the distinguished manners of the early Muslims.
NB: For further detail on the rulings related to "the Manners of Adornment & Attire" see Chapter 15: "The Book of Manners" by Fu'ad Ibn Abdul -Azeez Ash-Shulhoob