Is there a certain amount of miles an officer has to pull you over from the time of a traffic violation?

No. If the Officer has probable cause to believe that a violation of the Vehicle Code has occurred, he can initiate a traffic stop. The location of the traffic stop is usually irrelevant (unless outside of jurisdiction).

Based upon the facts in your question, one of two things occurred.
1. The Officer was attempting to gather evidence to determine if you were DUI or
2. The Officer was delaying the stop for officer safety reasons, either to locate an appropriate safe spot to initiate the stop or waiting for a backup officer to arrive.

I would venture to guess that he was simply trying to gather evidence of a DUI.