Must I carry my Ontario driver’s licence with me at all times?

At any time you are driving a vehicle, yes, always carry your driving licence to prove you have the privilege to drive.