Computer turns on but not off

I built a pc from some parts I had laying around and new ones I bought. When I went to turn it on it starts up normal for a fraction of a second and the the fans go to full speed and it won't boot even into bios and it won't turn off unless you unplug it.I've tried removing the hard drive, graphics card and ram. When the ram is removed it doesn't beep or anything just spins up.