Vertical bar with rm

Hello,

1. I want to remove everything except test dirctory, why the following command don't work:
ls |grep -v test |rm -fr

2. According to manual, vertical bar get current program result and add it into the last position of next command, how to make the result into mid of next command, is there any auto variable can use?