Sergei Bobrovsky stopped 28 shots to tie his career high with his 32nd win. Josh Anderson also scored and Oliver Bjorkstrand had two assists to help the Blue Jackets win for the fourth time in five games.
Columbus won for the second time in two days after its bye week and moved past the Rangers into third place in the Metropolitan Division.
The Blue Jackets won three of five in the season series – including both games at Madison Square Garden – after losing seven straight meetings coming in.
New York lost for just the third time this month (9-2-1) and had its four-game point streak snapped (3-0-1). The Rangers, who came in 5-0-1 in their last six at home, were playing for the fifth time in six days – with the previous three games going beyond regulation.
With Columbus leading 2-1 after one period, Lundqvist made a nice stop on a breakaway by Atkinson to keep it a one-goal game just past the 9-minute mark of the second.
However, seconds after Nick Holden made a sliding save of Zach Werenski‘s shot at an open net with Lundqvist out of position, Wennberg scored off a rebound of a shot by Bjorkstrand for his second of the game and 12th of the season to make it 3-1 at 9:47.
Atkinson got his second of the game and career-high 29th at 4:12 of the third. He took a pass from Brandon Dubinsky, skated in and beat Lundqvist on the stick side. Atkinson also tied his career high of 53 points set last season.
Anderson scored from in front in the final minute for his 14th, and Fast scored just before the final buzzer.
Wennberg got the Blue Jackets on the board 1:02 into the game when he deflected David Savard‘s shot down out of the air and past Lundqvist for his 11th.
The Rangers tied it when Nash received a no-look pass from Jimmy Vesey and fired it from the right circle over Bobrovsky’s glove at 4:07. Blue Jackets coach John Tortorella challenged for offside, but the goal stood after a review.
Columbus went ahead 2-1 with its second power-play goal in 14 games. With the Rangers’ Adam Clendening off for roughing, Atkinson wristed a shot from the center point past Lundqvist at 6:47.
Nick Foligno was sent off for holding with 2:47 left in the first, putting the Blue Jackets short-handed for the first time in three games. Bobrovsky stopped great power-play chances for Brandon Pirri and Mats Zuccarello.
Columbus’ Brandon Saad was awarded a penalty shot when he was held before he could get a shot off during a short-handed breakaway in the final minute, but Lundqvist deflected his attempt away.