Steve Evans described supporting his Millers as being like a roller-coaster ride after they swapped five goals with Cheltenham in 31 first-half minutes.
Daniel Nardiello's double and a goal for Ben Pringle had his side 3-2 up with just over half an hour on the clock, before man of the match Lee Frecklington finally settled the goal-fest with Rotherham's fourth in added time.
"We have a habit of making things difficult for ourselves and after going a goal down so quickly we showed some character," Evans said.
"It was a big performance. We played some special stuff in periods. It was a pity we had to wait so long for the fourth goal because I think our performance deserved a more comfortable win.
"When I joined this football club I said it was going to be a roller-coaster ride, but I didn't realise it would be 3,000 feet high. We said to the boys we were going to play with pride and passion, but what a response we gave the supporters and what a response they gave us back.
"We were a goal down after 50 seconds, and I said to my staff on the bench, listen to the supporters, and that was for them. We were much better today in every department and I am really, really pleased because there were some big performances."
Cheltenham boss Mark Yates, who saw Chris Zebroski continue his recent goalscoring form with another strike, was not a happy man after the game but paid tribute to the role played by Keith Lowe who opened the scoring within the first minute.
"We had to chase the game," he said. "We needed to stand up because we knew there would be a backlash from Rotherham. It was far from pleasing, but the good thing is we have another game on Tuesday.
"There was a lot I wasn't happy with today. There weren't too many positives to come out of the game. Keith Lowe was outstanding but apart from that I don't think too many got any positives out of the game and that's disappointing.
"The subs came on and livened it up but we knew we had to stand up and be counted and today we didn't in aspects we are quite good at and pride ourselves in. We played some decent football but if you give teams goals and look wide open you are going to get caught. It was far from pleasing."