I bought two of these second hand a while back. One connects to Windows with no problem. The other gave me "The last USB device connected was not recognized by Windows". I tried the pinhole reset, Settings factory reset, none of that worked. I tried pulling all other USB devices, tried different USB ports (but why, when the other one worked perfectly doing none of those things?). I was about to throw it out, and I decided to take it apart just out of curiosity. I took out those two little screws on either side of the USB port, then carefully pried off the top half of the two-part shell. Then, just on a whim, I connected it to the PC like that, top cover off. WHAT?? It connected properly and I was able to fill it with MP3 files! When I was done, I put the top cover back on, put the two screws back in and tightened them normally (i.e. not over-tightening). Tried connecting it like that, and once again, it could not connect. I took the screws out and this time left the cover on. It connected. OK, now this time I untightened the screws but left them in the screw holes. It connected.
So my conclusion is there is some pressure being exerted on the USB port when the screws are in there, tightened correctly, and possibly one of the two data connectors (send, receive) isn't in proper contact with the corresponding cable plug. Try loosening those two screws with a jeweler's screwdriver, leaving them in the holes (so they don't fall and get lost). It's worth a try if nothing else worked.