Aston Villa moved out of the relegation zone by drawing against a West Ham side who finished with 10 men after Arthur Masuaku's red card
West Ham boss Manuel Pellegrini disagreed with the decision to send off his left-back.
Eddie Howe said Bournemouth's victory over Everton was a "huge moment in our season" as Callum Wilson scored twice at Vitality Stadium
"Psychologically, it was hugely important to win that first home game which is why today was a huge moment in our season," said Bournemouth manager Howe.
Norwich City produced a magnificent all-round display to inflict Manchester City's first Premier League defeat since January at an ecstatic Carrow Road.
Norwich City manager Daniel Farke, speaking to BBC Sport: "Of course it is a special day for us and for the club, against one of the best teams in the world and when we have so many injuries.
Manchester United returned to winning ways and ended Leicester's unbeaten start to the season by edging to a Premier League victory in a pulsating contest at Old Trafford.
After thrashing Chelsea 4-0 on the opening day, United had failed to win any of their last three games but took all the points to inflict Leicester's first defeat of the campaign.
Premier League leaders Liverpool shrugged off an early scare to sweep past Newcastle and maintain their 100% start to the season with another dominant attacking display.
Liverpool manager Jurgen Klopp: "I started enjoying it after 35 minutes when we arrived in the game.
Sheffield United boss Chris Wilder says his side "have to wise up quickly" after they were beaten by Southampton in a tight game at Bramall Lane.
The Blades have now won just one of their opening five games on their Premier League return.
Tammy Abraham scored a brilliant hat-trick - and an own goal - as Chelsea destroyed Wolves at Molineux. A day to remember for Abraham, who is joint top of the Premier League goalscoring charts alongside Manchester City's Sergio Aguero
The England forward scored twice in the space of seven first-half minutes after Fikayo Tomori - another 21-year-old - had opened the scoring with a stunning 25-yard finish.
Arsenal staged a stirring comeback to earn a point in a chaotic, thrilling north London derby against Tottenham at Emirates Stadium.
Arsenal manager Unai Emery, speaking to BBC Sport: "It was an amazing match.
Liverpool manager Jurgen Klopp played down striker Sadio Mane's reaction after being substituted in the 3-0 win at Burnley, and then praised his forward line for helping the club achieve a 13-match winning run in the Premier League.
Jannik Vestergaard's towering header gave 10-man Southampton a point after another moment of magic from Manchester United new boy Daniel James looked to have put them on course for victory at St Mary's.
Sergio Aguero was labelled a "special talent" by Manchester City boss Pep Guardiola after the Argentine hit two goals against Brighton, continuing his record of scoring in all four of his team's Premier League games this season
"Of course we need his goals, everybody knows it.
Jamie Vardy scored twice and made the other as an impressive Leicester maintained their unbeaten Premier League start by easing past Bournemouth at King Power Stadium.
Brendan Rodgers' side have won two and drawn two of their opening four games, showcasing their early ambitions of breaking into the top six at the end of the season with another excellent showing.