Given a point O and angle θ, a reflection is a transformation in which every point P with position angle α is moved to the point P′ such that PP′ has its midpoint on the line defined by the angle θ and that PP′ is perpendicular to this same line of reflection.
To express reflection as a linear transformation, we must map the origin to itself. Thus, we will assume that the origin lies on the line of reflection. If this is not the case for your transformation, you should translate so that this holds, and then apply another translation that takes the problem back to the original center point.
For now, we will just work with two dimensions. This is oversimplified, but it works for a general understanding.
Let P=(r,α) in polar coordinates. Then P′=(r,2θ−α). To verify this, note that since O lies on the line, we have OP=OP′. Also, the average of the angles of P and P′ must be the angle for the line of reflection, which is θ.
For matrix representations, we will have to convert to Cartesian coordinates:
P=(rcosα,rsinα)
P′=(rcos(α+θ),rsin(α+θ))
Using the angle addition formulas for P′,
P′=(rcos(2θ−α),rsin(2θ−α))
P′=(r(cosαcos(2θ)+sinαsin(2θ)),r(cosαsin(2θ)−sinαcos(2θ)))
Suppose the rotation matrix is as follows:
A=[acbd]
Thus, for AP=P′, we have the following equations:
arcosα+brsinα=r(cosαcos(2θ)+sinαsin(2θ))
crcosα+drsinα=r(cosαsin(2θ)−sinαcos(2θ))
Cancelling out the r factors and solving, we have:
a=cos(2θ)
b=sin(2θ)
c=sin(2θ)
d=−cos(2θ)
Thus, here is the matrix for a reflection across a line defined by angle θ:
A=[cos(2θ)sin(2θ)sin(2θ)−cos(2θ)]