Yes! If you have a category \$$\mathcal{C}\$$ with one object and with all morphisms invertible, it's secretly just a group. And if you have a functor \$$F: \mathcal{C} \to \mathbf{Set}\$$, it gives a set and an action of your group as permutations of that set.
For example, your set could be the 4 corners of the square, or the set of all points in the square. Your category \$$\mathcal{C}\$$ could be the one in Puzzle 107. The functor \$$F : \mathcal{C} \to \mathbf{Set}\$$ sends the one object \$$z\$$ in \$$\mathcal{C}\$$ to your set. It sends the four morphism \$$1_z, s, s^2, s^3\$$ in this category to the four rotations of your set!
More generally, for any \$$\mathcal{C}\$$ , a functor \$$F: \mathcal{C} \to \mathbf{Set}\$$ gives you a bunch of sets (one for each object in your category) and functions (one for each morphism) obeying the same relations as the morphisms in your category. So, it gives a "concrete picture" of the abstract category \$$\mathcal{C}\$$.