I've just realised the white/grey/black picture I mentioned above only works if \\(f\\) is _surjective_.
If that isn't the case, then there are a bunch of \\(y\\)s that aren't hit by anything in \\(X\\), and they correspond to empty partitions of \\(X\\). It's not clear what "colour" these empty partitions should be. In a sense they are also "borderline cases" but in the opposite way to the grey cells.
I suppose you could colour them "antigrey" and say \\(f_*\\) is grey and black, while \\(f_!\\) is antigrey and black. Then the double complement still works by flipping black and white, then complementing – which takes us from grey + black to grey + white to antigrey + black.
Note in particular that \\(f_!(S)\subseteq f_*(S)\\) only holds in the surjective case.
One more warning – the notation in Seven Sketches is the other way round (p25) – they use \\(f_!\\) for the direct image and \\(f_*\\) for the right-adjoint-to-preimage-that-doesn't-have-a-name.