- Do I need to sign up for Azure to use the REST API?
No, register an app with Azure AD is free, I follow the tutorial without any charge on Azure.
- Do I need to pay Azure to use the PowerBI REST API (in this case would it be PowerBI PRO + Azure)?
You only pay for pro license.
- Can I reassemble my dashboard (like on the web) using REST API?
What do you mean reassemble. Power BI REST API now supports embbeding a dashboard in your web application.
- How does this work (basically) a mix of REST API with embed / iframe?
Check this tutorial and the samples in Github.