Yunus Musah and Hugo Duro fired the visitors into a 2-0 lead by the 44th minute but Matheus Cunha pulled one back in the 64th.
Then, in stoppage time, Angel Correa and Mario Hermoso scored to give Los Rojiblancos and absolutely crucial victory.
The result means that Atletico hold on their place in the top four of La Liga and can’t be supplanted by Real Sociedad.
But it all worked out in the end. Atletico secured a thrilling comeback victory and Diego Simeone’s decision was vindicated.
However, questions do need to be asked of whether Felix’s future lies at the Wanda Metropolitano or not.
The Portuguese has been linked with a move to Manchester City, a team that play a brand of football more suited to his game.