Arsenal endured a dismal start to the new Premier League season when Christian Benteke struck twice to lift Aston Villa to a shock 3-1 victory at the Emirates where the home side finished with 10 men on Saturday.
Liverpool striker Daniel Sturridge scored and new goalkeeper Simon Mignolet saved a late penalty to earn their side a deserved 1-0 win over Stoke City in the early kickoff at Anfield and West Ham United beat promoted Cardiff City 2-0.
Arsenal took an early lead through forward Olivier Giroud but Benteke equalised by scoring from the rebound after having a penalty saved and the powerful Belgium striker fired home another spot-kick just past the hour mark.
Arsenal defender Laurent Koscielny was sent off for a second bookable offence before Villa fullback Antonio Luna grabbed a late third as the home fans left the ground muttering darkly about manager Arsene Wenger's failure to add to his squad.
Manchester United begin their title defence under new manager David Moyes at Swansea City in the late kickoff while Chelsea host Hull on Sunday with Jose Mourinho back in charge.
Sturridge put Liverpool ahead against Stoke with a firm 20-metre shot after 37 minutes and the hosts dominated a frenetic match, only to be denied a second by a brilliant goalkeeping display from Stoke's Asmir Begovic.
The visitors threatened occasionally on the break and were awarded a penalty when Liverpool defender Daniel Agger needlessly handled a long Charlie Adam free kick.
But Mignolet, who had endured a nervous debut, palmed away a weak effort from the spot by Jonathan Walters a minute from time to spark wild celebrations around the ground.
"That was a great performance from us," Liverpool manager Brendan Rodgers told reporters. "Big credit to Stoke and their manager Mark Hughes. They have come here and played well."
Joe Cole scored an early goal for West Ham and fellow midfielder Kevin Nolan added a late second to give Welsh side Cardiff a losing start in the top flight.
Norwich City and Everton drew 2-2 in an entertaining match at Carrow Road where, after a goalless first half, defender Steven Whittaker put the home side ahead before the visitors equalised with a superb shot by 19-year-old midfielder Ross Barkley.
Defender Seamus Coleman put Everton in front but Norwich striker Ricky van Wolfswinkel, the club's record signing, earned his team a point with a neat header on his debut 19 minutes from time.
Southampton striker Rickie Lambert, who scored a late winner for England against Scotland on Wednesday, converted a last-minute penalty to seal a 1-0 win at West Bromwich Albion.
Fulham won 1-0 at Sunderland thanks to a 52nd-minute header by midfielder Pajtim Kasami, his first goal for the club.