From what you said, I assume you are using Power BI Embedded and not using embedded reports in PowerBI.com. I also assume that database schema will be the same across all the different customer databases.
Yes, I would use the one PBIX project per customer approach. If the only things that changes between custom projects is the connection string and login info, you might be able to automated the whole process of adding a new customer.
1) Create a new Power BI workspace in your workspace collection in Azure
2) Upload a copy of the master PBIX file to this new Power BI workspace Azure
3) Patch the database connection string and the data source credentials
These steps can be automated using either C# or PowerShell. You will also have to figure out how to get the web app with embedded reports to load the correct reports for the current customer. This will be done by tracking the GUID for the Power BI workspace that is specific to that customer.