Back in India, his family travels to London to meet him and important occasions are celebrated together, of course, outside India.
Thus Modi Senior recently celebrated his 70th birthday abroad so that his son could attend.
The family says Modi certainly wants to come back to India - which is evident in his tweets as well - but does not reveal when that will happen.
"He has had to deal with a lot in the last two years but there's no doubt that he will come back bigger and stronger," says his father, who has supported him publicly and remains confident that his son will be cleared of the charges.
But at the moment, things aren't looking good for Modi.
Last week in London, a court declared him bankrupt in a case involving unpaid bills of about Rs 53 lakh. And he recently lost a libel case to former New Zealand cricketer Chris Cairns.
Modi had tweeted about Cairns being involved in match-fixing in the Indian Cricket League, once a rival of IPL. It is learnt that BCCI vice-president Arun Jaitley has prepared a report on Modi.