Actually, ignore my last reply. After looking more into how RLS works with Power BI embedded, I figured out that I could actually pass in a CustomerId as the username on the embed token, and then use DAX with the USERNAME() function to tie that to the CustomerId on the report.
So essentially, in RLS, I have a role "Customer" with the following DAX expression:
[CustomerId] = USERNAME()
Then, I created a token with the appropriate role and username:
{ "wid":"{workspace id}", "rid":"{report id}", /* other properties */ "roles":"Customer", "username":"my-customer-uid" }
I'm not sure if this is what you intended when you suggested using RLS, but it should definitely work.