I would suggest investing in a good machinist square. You have to have at least one truly flat surface. My go to for quite a while was a Starrett 6" rule or blade from a Starrett 6" try square. I kept it in a case and it's original cardboard sleeve so it wouldn't get dinged up. It's great you are next to a window, good light is a necessity. I also avoid tilting a straight edge for checking light unless you try it opposite as it can be slightly bent and give you false readings. Small granite surface plates are fairly cheap and are invaluable for checking your work. The side of a caliper was never meant to be used as a true straight edge IMHO.
Problem for me is after roughing in I do better rubbing the work on the file held on a flat surface than holding the work in a vise and filing it. There is a reason the final step for true flatness is to lap. It's hard to file truly flat. Another truly necessary tool is a set of tool room stones to stone your blade/rule because the slightest ding that you can barely feel will cause all kinds of light between the blade and surface giving another false reading. It's very attainable to get to .001 with very careful attention. But to get beyond that is many times harder.