Think of it this way: there is never a doer, even when thought says there is one. This knowledge can set you free. Don’t worry if it seems like you’re a doer. It’s just like the sun rising and setting – it seems like the sun is orbiting the earth, but the knowledge is there telling us that it is the earth that is orbiting the sun.
Don’t worry too much about the appearances or whether or not you get lost in thoughts. Sometimes, in fact very often, thoughts needs to express themselves, and in expressing themselves you can learn about yourself on the human level/level of the mind. A bit of worry can be quite good, as it helps us prepare for future events.