According to anonymous sources cited by news-agency ANI, Garg has already applied for a voluntary retirement.
According to rules, he will have to serve a notice period up to three months or until his application for voluntary retirement is accepted.
On Wednesday, Atanu Chakraborty, a 1985-batch Gujarat-cadre Indian Administrative Service (IAS) officer, was named as Garg's replacement at the Ministry of Finance. The government reshuffled top bureaucrats across 12 ministries, departments and agencies.
The rejig came less than three weeks after the Union Budget was presented in the Lok Sabha and a day after it was passed by Parliament.
The changes would come into effect immediately.
Garg, who was also the Secretary, Department of Economic Affairs, will replace Ajay Kumar Bhalla as the Power Secretary, an official notification said. Bhalla has been appointed as the Officer on Special Duty (OSD) in the Ministry of Home Affairs.
This change is seen as a relegation for Garg, as the designation of Finance Secretary is one of the most coveted in the bureaucratic circles and goes to usually the senior most officials of the ministry.
The move, however, should not be surprising as this is not the first time the Narendra Modi-led NDA government has made such major changes.
In 2014, after the Modi-led government took charge for the first time, government transferred the then Finance Secretary Arvind Mayaram to the tourism department.
With Garg out of the Finance Ministry, the government will now have to choose between the current revenue secretary Ajay Bhushan Pandey and financial services secretary Rajiv Kumar to be given the position of the finance secretary. Interestingly, both the officers are senior-most in the ministry joining the Indian Administrative Service (IAS) on August 21, 1984. However, Kumar is senior to Pandey on age. Government will decide on this while appointing the next finance secretary.
As per the Wednesday evening order, the next DIPAM Secretary will be Anil Kumar Khachi. He is a 1986 batch IAS officer from Himachal Pradesh cadre.
Another major appointment was that of Anshu Prakash, the Additional Secretary, Department of Telecommunications (DoT), as the Secretary of the deparment. He would take charge July 31 after the superannuation of the current DoT secretary Aruna Sundararajan.
Further, the cabinet committee on appointments also approved the appointment of Guruprasad Mohapatra, the current Chairman, Airports Authority of India as the Secretary, Department for Promotion of Industry and Internal Trade (DPIIT), Ministry of Commerce & Industry.
R.S. Shukla has been appointed the Secretary, Ministry of Parliamentary Affairs against the existing vacancy, the notification said. Also, Anuradha Mitra has been appointed secretary (department of official languages) in the home ministry while Ravi Kapoor has been appointed textiles secretary.
Two more secretaries were appointed by the government including Atul Chaturvedi as secretary, animal husbandry and dairy and P.D. Vaghela as pharmaceutical secretary.
The government also upgraded 11 oficials to the level of special secretary.
With Agency Inputs