But Arsene Wenger's men -- who haven't won a major trophy since lifting the FA Cup in 2005 -- turned the table at the Emirates Stadium.
Alex Oxlade-Chamberlain gave the Gunners a 16th minute lead when he followed up after Steven Gerrard blocked Yaya Sanogo's initial shot.
Oxlade-Chamberlain turned provider in the 47th minute, outpacing Daniel Agger before a pull-back for Lukas Podoslki allowed the Germany star to fire them into a 2-0 lead.
But Liverpool captain Gerrard ensured a nervous finish for the London club when he scored from the penalty spot in the 59th minute after Podolski fouled Luis Suarez.
The Gunners will now face Everton in the quarter-finals after the Toffees beat Swansea 3-1 earlier on Sunday.
Everton manager Roberto Martinez, who won the Cup with Wigan last season, saw Lacina Traore, his loan transfer signing from Monaco in January, take just four minutes of his debut to give the Merseysiders the lead.
Jonathan de Guzman drew Swansea level in the 15th before second half substitute Steven Naismith put Everton in front again -- with his fourth goal in six games -- and then won the penalty from which Leighton Baines scored to put the result beyond doubt.
However, there was a worrying moment for Everton when, shortly before full-time, Naismith went off with concussion after being caught in the head by Jordi Amat.
The draw took place just after the final whistle, with Martinez's attention turning towards the Emirates.
"The draw hasn't been too kind in terms of playing away from home, but we will embrace it and look forward to it," Martinez told ITV Sport.
As for Naismith, he added: "We are a little bit concerned by his concussion."
Naismith came on for Traore and Martinez was pleased with the striker's display.
"It's important," he said. "Lacina has been working very well. We needed to introduce him slowly. It was pleasing to see him on the pitch and see him scoring."
Sunday's other Cup tie saw third tier Sheffield United beat second-tier Nottingham Forest 3-1.
United, managed by former Forest favourite Nigel Clough, could be involved in a derby quarter-final against Sheffield Wednesday if their city rivals beat Charlton Athletic in a re-arranged match.
Jamie Paterson headed Forest into an early lead before the on-loan Conor Coady drew the Blades level in the 66th minute.
And Chris Porter then sparked scenes of near delirium amongst the Bramall Lane faithful with two goals right at the end of the match.
He put them in front with a 90th minute penalty following Greg Halford's hand-ball and then sealed victory in stoppage-time with a finish from close range.
The last eight line-up also saw FA Cup holders Wigan paired against Manchester City in a repeat of last season's final.