Am I Just Fat???

Well...here we go again...I'm jumping on the diet wagon. I'm so done with weight being a constant battle in my life, a battle that begun in my childhood. Back then I was known as, "Chubby" I used to think it was a result of my mom feeding us frozen breakfast, lunch, and dinners. She cooked until being diagnosed with Hypoglycemia and I think that cooking dinners for us and then cooking a special meal for herself became too much so we did the frozen meal thing. My brother, ate the same diet as me (plus some) but he was always in perfect shape with no fat on him. He wasn't ever really too skinny...he was just always normal. But then there was chubby me.
Then, later in my life my weight went something like this:
Gained weight...more... and more...dieted....lost weight...lost more...steady...gained...gained..gained...dieted...lost...lost...lost...steady...steady...gained...gained...gained..dieted...well...you get the picture.
I've tried (in no particular order)  fitness centers, Yoga classes, diet classes, fat free, low carb, starving myself, diet pills, diet drops.  One thing I can say is that they all work. They work just as well as any kind of self help smoking aid would...it helps you become the person you want to be by kicking the habit (or in my case the fat) but when it's time for you to maintain and go out there on your own...you sink.
One thing that I find funny is that I'm not an unhealthy eater. I know when people look at me and see how overweight I am they are thinking I'm someone who sits around and eats crap all day. That would be incorrect.
No, I do not "workout" but I like to think that running after my children, running around the house all day, cleaning, loading firewood, kindling, bricks, gardening, swimming, walking, pulling weeds could be considered as active...?
I just can't figure out how I've gained all of this weight and the one reason I've had for not wanting to go on a diet is because I know I'll end up exactly where I began.
I wish I could have someone follow me for a month and take note on my activities...ok..maybe not ALL my activities...and then also my eating habits and tell me what's wrong. I've had blood work done...nothing wrong there. I wish I could snap my finger and make this all go away.