West Brom will play in the Premier League next season, where they could yet be joined by Swansea City, but Wigan Athletic are in League One.
With Brentford having faltered earlier, Fulham beat Sheffield Wednesday to keep their top-two hopes alive.
An eighth consecutive win for Championship high-flyers Brentford kept the pressure on Leeds United and West Brom in the race for promotion.
Ethan Pinnock's 85th-minute header secured Brentford a sixth successive league win, closing the gap on the top two in the table.
It was not a game to live long in the memory but Millwall downed Charlton thanks to Jake Cooper's 81st-minute striker.
West Brom dominated against Birmingham but could not score, while Louie Sibley and Rhian Brewster produced heroics for Derby and Swansea.
Arsenal made an impressive return to action with an emphatic win over Charlton Atheltic at the Emirates Stadium.
Jonathan Leko took aim at the FA and other bodies for their handling of Kiko Casilla's racial abuse charge, despite the Spaniard's ban.
The FA has punished Kiko Casilla for abusing Jonathan Leko "with reference to race and/or colour and/or ethnic origin".
After West Brom won on Tuesday, second-placed Leeds United moved back within four points of the Championship pacesetters.