My 2 cents -
Five adults are pretty tight, unless one is willing to sit or stand in the cabin - a lot (this is where a custom cabin seat would come in handy).
I love to sail with just 2 (my wife and I), but I also routinely take 4 adults sailing. It works well as long as everyone does their assigned duty
To accommodate 4 adults each person has an assigned seat, and assigned duties while under way. Everyone has to be familiar with their roles and the boat, and no one gets a free ride.
Typically I sit starboard, rear of the cockpit. From there I captain the boat, and manage the auxiliary trolling motor to get in and out of marinas, and for docking.
My wife sits port, rear of cockpit. On a port tack, I handle the tiller, she handles the main sheet. On a Starboard tack, she handles the tiller, I handle the main sheet. No moving around or switching sides as we come about during tacking.
Adult guest number 1 (usually my son) handles the starboard forward sitting position in the cockpit, and manages the main halyards (peak and throat), and the starboard jib sheet
Adult guest number 2 (usually my DIL) handles the port forward cockpit location, and only has to handle the port jib sheet, and the black line that retracts the jib when we need to reel it in or deploy it. That position is also assigned to be the 'gopher' for anything needed that is stored in the cabin (drinks, lunch, suntan lotion, seat pads, etc.)
No one moves around much during the sail, and everyone is involved in coordinating handling the boat through any tacking maneuvers.
The 2 in front handle any raising or lowering of the main sail, and when lowering it, they are tasked with handling the halyards, and securing the sail material as the arm comes down, and using bungee cords to secure the main from flapping around. (I don't have Lazy Jacks installed yet). I manage the tiller when raising or lowering the main (wife is not too good at keeping up pointed into the wind for some reason).
When docking, either my son or DIL are charged with jumping out onto the dock since they are in a perfect position to jump out onto the dock as we arrive, and securing the boat to the dock cleats, and they are both young enough to jump onto the dock from a slowly moving boat. They also cast us off and have to jump in as the boat moves away from the dock.
Lots of team work, and everyone needs to understand their role, and work primarily within the space they are assigned
Downside is a new comer takes some training and explanation to understand their role and how the crew has to function together.
I would think that if you have a young family, the children would be thrilled to have an assigned seat, assigned duties and you'd maybe have some great family times as you work together to handle navigating an afternoon sail.