This paper presents a scheme for video denoising by diffusion of gray levels in the video domain, based on the Computational Algebraic Topology (CAT) image model. Contrary to usual approaches, using the heat transfer PDE and discretizing and solving it by a purely mathematical process, our approach considers the global expression of the heat transfer and decomposes it into elementary physical laws. Some of these laws link global quantities, integrated on some domains. They are called conservative relations, and lead to error-free expressions. The other laws depend on metric quantitites and require approximations to be expressed in this scheme. However, as every step of the resolution process has a physical interpretation, the approximations can be chosen wisely depending of the wanted behavior of the algorithm. We propose in this paper a nonlinear diffusion algorithm based on the extension to video of an existing 2D algorithm thanks to the flexibility of the topological support. After recalling the physical model for diffusion and the decomposition into basic laws, these laws are modeled in the CAT image model, yielding a numerical scheme. Finally, this model is validated with experimental results and extensions of this work are proposed.