Louis van Gaal was left frustrated after his side threw away two leads
At least Louis Van Gaal was not bored as his men led 2-0 then Newcastle clawed their way back into it before Wayne Rooney scored with a stunning drive, a strike that even brought a smile to his manager's face which quickly changed with left-back Dummett's 90th minute strike.
"It does feel like a defeat," said Rooney. "After scoring three good goals, it was maybe a lack of concentration – not staying with runners for the third – and that is disappointing.
"If you take a two-goal lead, we should be taking all three points and killing the game off. We didn't take all our chances and we have been punished for it.
"We always want to try and play entertaining attacking football that is what we work on and try and do, and hopefully we can come up with more solutions."
While Rooney thought it was more like, Dummett was totally the opposite.
He said: "It definitely feels like a win, especially with being 2-2 and conceding right near the end. When you're near the bottom you need every point you can get."
Aleksandar Mitrovic put the hosts level at 2-2 with a disputed penalty to match an equally contentious early spot kick converted by Rooney to open the scoring after Chancel Mbemba's handball.
Van Gaal said: "I'm very disappointed, we're very disappointed. We have to blame ourselves because we could have finished the match much earlier. I've said that to my players. You can say the referee makes a questionable decision for their penalty, but we could have finished it much earlier.
"It was luck for Newcastle United, but we had big chances and we should have taken them. In my opinion it's no discussion on our penalty, Mbemba is putting his hand to the ball, he did it with purpose so it's a penalty.
"For their penalty, it's heavyweight wrestling between Mitrovic and Chris Smalling. You can see he's pushing the head of Smalling to the ground. Then they are heavyweight champions and you can judge who is the champion.
"The referee makes his decision and we have to accept it. We've scored three goals and got one point for it. We had to win today, but that's football. That's also why people love to go to a match like this."
Newcastle manager Steve McClaren said: "It was a magnificent response from the players. We could have gone down after falling 2-0 behind, but credit to them we kept fighting.
"We showed what we're all about here. We've shown that character and attitude and quality. We started well and actually had more of the ball than Manchester United, who are usually way out there in the possession stats.
"We've gone behind, we've come back and we've deserved to equalise at the end."