Vice-captain Raghunath struck in the 22 minute but only to see Japan equalise in the 50th minute though a stroke. Raghu again brought India on top in the 59th minute. Gurwinder Chandi posted a goal in between in the 51st minute.
India had defeated China in the opener 4-0 yesterday with Raghunath finding the net twice.
Despite territorial dominance, India could not move the scoreboard well due to poor shooting and soft attempts at goal. It took 22 minutes for the Indians to break the logjam.
Raghunath struck from second penalty corner to put India ahead.
Japan equalised in the second half through Koji Kayukawa (50th min), but star striker of India Gurwinder replied in style in the next minute to put India again on top.
India earned three penalty corners on the trot in the last 10 minutes of which again Raghunath converted the third to place India comfortably.
Japan goalkeeper Akira Morishima had an outstanding performance under the cage today and made the task of India very difficult.
India took Japan easily in the first half and when showed enormous urgency in the second half, it got the dividends.
Indian chief coach Michael Nobbs praised the Japanese for the good fight they put up.
"They played really very well. Our defense was holding on to the ball for too long. We have to work on that, we don't play that way, that's not our style," the Australian said.
"We committed a lot of silly mistakes, it cost us. In fact we should have got 4, 5, goals more. I think we played easy," Nobbs added.
Nobbs also disputed the award of stroke to the Japanese side, helping the rivals to equalise.
"Not much to say about umpiring. The stroke was not a correct one. The umpiring is really poor standard, the stroke wasn't a correct one. Injuries will be there in the tournament, today Sunil got injured. It will be a very tough game against Malaysia," he said.
The losing team coach Kang Keon Wook too praised his team. They struck to the plan. India's strength was penalty corner, they proved it again, he said.