Yeah I'm not sure if that's possible. What's interesting is if you enable drill down and click on a visual, under "visual filter" it says that it's been applied. However, clicking on a visual without drill down enabled doesn't add a "filter" to any of the filter levels on the right side pane (visual, page or report).
If clicking on a part of visual changes everything else on a page, you would think that's a "page" filter, albeit a temporary one as once you click out of the visual, the filter should be removed.
Just a couple of other thoughts is all...