I think you might need the flexibility of the constant to get adjoint maps that aren't inverses.