The emerging memristive Memory Processing Unit (mMPU) overcomes the memory wall through memristive devices that unite storage and logic for real processing-in-memory (PIM) systems. This paper vastly accelerates the fundamental operations of matrix-vector multiplication and convolution in the mMPU. We overcome the inherent asymmetry limitation in the previous solutions by utilizing techniques from block matrix multiplication. We present the first fast in-memory binary matrix-vector multiplication algorithm by utilizing a tree-based popcount reduction. For convolution, we present a novel in-memory input-parallel concept which we utilize for a full-precision algorithm that overcomes the asymmetry limitation in convolution, and a fast binary algorithm.