I considered a longer boat, and the use of an outrigger befor I built the skiff. My goals were a simple, easy to paddle, stable enough for my large size, fit in my truck by myself, boat to fish out of. A longer boat with outrigger would meet the goals of stability and easy to paddle, but would be a compromise of simplicity and fitting into my truck. My skiff is simple to use, stable, easy to load, and I didn't have to remove my tool box for it to fit in my truck. It is not "hard" to paddle, but after leisurely paddling and fishing all mourning with a tail wind, the return paddle into a stronger wind could have been easier. I am not an experienced paddler. The boat may be easy. I have no experience base to know. Your comment about it's pace and the head wind trip makes me believe it is not.
I'm trying to decide what and how much any changes will be, befor doing all work.
How wide was the bottom of the boat at the transom? It may be I can have a narrower transom, keep the same length of boat without too much loss of stability.