Short answer: Yes, you can.
But there is even a better way! You get your PHP files to be included automatically when you instantiate the class that is defined in that file. This is called autoloading. [Wikipedia]. Just to bring you and other PHP developers up to speed with best practices in modern PHP development; Take a look at composer and Dependency Injection. It provides a way to lazy load services so that they only get loaded when you need them.
With Composer all your dependencies will be managed and an autoloader will be created magically. You just need to put a single require_once __DIR__ . '/vendor/autoload.php';'
at the top of your code and from that point on, whenever you do new PhpMailer
for the first time it requires the PHPMailer class file. See the PHPMailer documentation for how to install PHPMailer using Composer.
It might seem a lot of work, compared to just typing require_once before any call to the PHPMailer, but in the long run, when your project grows, you will profit!
I also encourage you to read the above link on Dependency Injection since that might help you to even better decouple and structure your code. A good container to get started with might be PHP-DI. Good luck!