Caretaker boss Peter Shirtliff bemoaned his luck after Bury had a man sent off for the second game in a row as Doncaster's numerical advantage finally paid off at the Keepmoat Stadium.
Shakers skipper Steven Schumacher was dismissed for a second bookable offence in the 68th minute after tugging back Chris Brown, who then went on to seal a 2-1 victory for Rovers six minutes from time.
Schumacher had been booked in the first half when he tried to stop Kyle Bennett as he raced through on goal, but Shirtliff felt it was a harsh decision.
"I was annoyed about the first booking because there was a similar incident involving Doncaster that went unpunished and it just seemed to me there was a lack of consistency," he said.
"I thought we made a really good start to the game and we were unlucky to come away with nothing to show for our efforts. It was a very open game - far more than I expected - but that's just the way it panned out.
"We've got Coventry next who also came down from the Championship so it's been a tough start for us. But we can't feel sorry for ourselves. We have to pick ourselves up and have another go on Saturday."
Bury began in fine style and snatched the lead in the fifth minute when full-back Joe Skarz forced the ball home at the near post following a corner.
Rovers went close to equalising several times before Bennett latched on to a pass from Brown and held off his marker to slide home a low shot. Ten-man Bury battled bravely before Brown ended their stubborn resistance late on, beating the offside trap to fire home in fine style.
It was his third goal in three games after missing most of last season through injury and Rovers boss Dean Saunders said: "It's been a great start for Chris. He will take the plaudits and rightly so but it was a great team effort as well.
"We've got maximum points from two games and I'm delighted with that but it takes half-a-dozen matches before you know where you really stand."