Run any app off a USB drive?

Hey, guys, do you have the experience of running your daily used apps off a USB drive?

Which is I am now trying to do. But I find that only portable apps can be run like that.

while some apps like MS office are not portably available. How can I do that?