How does it affect my credit score being secondary signer?!?

Some credit reports list the secondary signer as also being responsible. Your insurance company probably checked those reports. If you did not sign the credit card application you really aren't responsible and should relay that information to the insurance company.