Solar magnetic fields are responsible for solar coronal mass ejections and other eruptive phenomena that govern space weather. Today, most of our knowledge about the solar magnetic field topologies was derived from the measurements of the magnetic field of the solar photosphere. The solar chromosphere is dilute, and associated magnetic fields are weak, which makes them difficult to measure. To address this challenging issue, we propose the use of the He I D3 5876 Å line, together with the He I 10830 Å line, to measure the off-limb weak and moderate chromospheric magnetic fields in the Hanle effect regime. We show that the current approach of using the polarization amplitude or degree cannot reliably retrieve the magnetic fields. We demonstrated that by using the Stokes linear polarization profiles simultaneously with the 5876 and 10830 Å lines, without the use of the circular polarization, chromospheric magnetic fields as low as and down to a few gauss can be measured easily and reliably with today’s instruments. We further demonstrate the potential to measure the weak chromospheric magnetic fields with the Stokes linear polarization profiles by using the He 5876 Å line alone, which has not been fully investigated yet. This work will provide a new technique for future daily synoptic observations for the weak magnetic fields in the solar chromosphere for both the active and quiescent regions, which are still difficult to measure.