A formulation of quantum electrodynamics is proposed in which all the propagators and field operators are gauge invariant. It is based on an old idea of Heisenberg and Euler which consists in the introduction of the linear integrals of potentials as arguments of the exponential functions. This method is generalized by an introduction of the so-called ''compensating currents'', which ensure local, i.e. in every point of space-time, charge conservation. The linear integral method is a particular case of that proposed in this paper. As the starting point we use quantum electrodynamics with a non-zero, small photon mass (Proca theory). It is shown that, due to the presence of the compensating current, the theory is fully renormalizable in Hilbert space with positive definite scalar product. The problem of the definition of the current operator is also briefly discussed.