Indeed Collider adds are queued up to happen in the next frame much like Entity and Component adds. This allows multiple Colliders/Entity/Component additions to happen in one frame keeping the state consistent so that the newly added objects work properly.
One possibility is to add all your colliders at creation time then change their physicsLayer to an unused layer. When you want to activate the collider change it's physicsLayer to the appropriate layer toggling the current Collider to the unused layer. You can then essentially toggle the colliders on/off at will.
I am personally not a big fan of doing things this way but it may work fine for your specific situation. Issues can arise when the Collider is "turned on" if it happens to be overlapping other Colliders. It requires a bit of defensive coding to keep things stable.
An alternative that may work for your situation (I use this a lot for stuff like punches, kicks, sword swings, fast actions that only last a few frames) is to not use a Collider directly at all and instead when the animation frame happens query the Physics class directly.