I was kind of thinking about it the other way, how to respond at the right time and tempo, after the count off, someone else counting off.
Practice.
I used to play in a Swing band with a conductor that had very little experience. We worked together in most of the count-offs. She would check with me the speed sometimes (I would respond with a rough tempo on the hi hat if she gave me the cue to do so) and would be looking at me when she counted off, if she saw my face contort then we'd abort!
What I learned though is that it's very important to practice the beginnings and ends of pieces. You absolutely have to know
exactly what you're going to play in the first four bars because you'll be concentrating on the rest of the band for that period, making sure your dynamic is right and that the band are following. There's no space to get it wrong in those first four bars - after that, you're settled in and the tempo can be more fluid, with the conductor - of course.
Usually, I would get two bar count-offs. One-Two, One-Two-Three-Four. Something of that ilk. I made sure I could see what was going on because we were responding to visual cues. The only major incident of a count-off going wrong was (sadly) in my last concert with that band, where I couldn't see the conductor (stage issues - fatal error, drummers always need to see the person that's counting off) and the count-off was far too slow. The first four bars were a disaster and without decent monitoring, it was very difficult to hear the band. Not my proudest moment. I was furious!
So really, it's about communication (however you're doing it) and knowing
exactly what is in those first few bars. If mistakes happen after that period, you can correct them but in the period where the time is being established with the rest of the band, you have to be absolutely sure about what you're playing and be able to respond immediately to any errors. It's usually better to go along with the 'wrong' speed for the first section of a piece and try to change it when there is a musical break, e.g. a chorus or particular cadence approaching. It's much less jarring but needs to be done gently. It's much easier to speed up than it is to slow down too - chorus sections are often more energised so it's easier to 'drive' towards them.
Just a few observations.