I'm doing "okay" in physics, should I stay in engineering?

I am an EE major. I don't know exactly how well I'm doing in physics 1, but I almost dropped for fear that I was failing (got a 40 on the first midterm, and a 57 on the second). My adviser said that the 57% was a good grade and that a lot of people are struggling with physics. My question is, if the curve of the cl allows me to take physics 2, how much of the material from physics 1 carries over to physics 2? Also I don't feel like I know the material well enough to p, even if a curve did allow me to. As an engineer, I'm sure that I will have to know a lot about physics, especially EE. What should I do? Thanks in advance.