This paper presents a novel optimal filtering approach that can be integrated into video compression schemes to improve video compression performance. In the proposed approach the encoder computes the coefficients of a linear filter in an optimal way, so as to minimize the squared error between the original frame and the filtered reconstructed frame. The encoder then multiplexes the filter coefficients into the bit-stream for decoder access. Reconstructed image frames are filtered at the decoder using optimal filter coefficients to obtain improved image frames. The optimal filtering approach is evaluated in the form of a post-process filter as well as in the form of an in-loop filter. It is shown using H.263+ and H.264/AVC that the proposed approach improves video compression performance of standard compression schemes.