PC has power but won't stay on

Hello,
About a month ago my PC froze so I turned it off and the next day when I went to turn it on and it wouldn't turn on. I order a new PSU, but that didn't seem to work, and I've found my orginal one works better. I sent back my motherboard asking for the manufacturers to fix the motherboard. On the website they have I was told there was a BIOS error and to replace the BIOS chipset yet I don't know if they did it or If i'm supposed to.
Anyways when I put the motherboard I got back into my PC and I go to turn it on it will only start for a second, yet if I hold the power button in it'll stay on but won't boot up. All fans get power, the PSU and GPU also get power along with the CPU fan. I've tried moving the RAM to different slots and I'm pretty sure that's not the problem.
I'm looking at the or the PSU being the problem. To be honest I have no idea what the problem is and what to do so any help or response I get will be greatly appreciated.
Thank you.

I have a Gigabyte GA-F2A85XM-D3H
A Seasonic S12Bronze PSU
Rosewill challenger case
AMD Quad-core processor
Zotac GTX 650 ti GPU